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ABSTRACT 


In  aviation,  spatial  awareness  and  spatial  orientation  are  essential  for  perfonning 
the  task  of  recovering  from  an  unusual  attitude.  Degraded  spatial  awareness,  particularly 
in  extreme  flight  situations,  may  lead  to  lower  operational  effectiveness  and  to  loss  of 
equipment  and,  in  manned  aviation,  loss  of  life.  Therefore,  improvements  in  spatial 
awareness  are  important  in  complex  3D  environments,  including  both  manned  and 
unmanned  aviation. 

The  main  goal  of  this  thesis  was  to  determine  whether  a  new  prototype  display 
design,  called  WEBER-Box,  is  a  useful  alternative  or  supplement  to  traditional  flight 
instruments  for  unmanned  aviation.  In  addition  we  combined  the  traditional  flight 
instrument  as  well  as  the  WEBER-Box  with  a  colored-coded  indication  when  the  aircraft 
entered  an  unusual  attitude. 

In  this  experiment,  the  participants  executed  typical  tasks  of  a  UAV-operator.  We 
investigated  the  influence  of  the  WEBER-Box  on  UAV  operator’s  orientation 
performance.  The  important  results  can  be  summarized  as  follows: 

1 .  significant  improvement  in  correctly  solving  the  orientation  tasks 

2.  significant  reduction  in  time  to  solve  orientation  tasks 

3.  color  coded  indication  of  unusual  attitude  significantly  decreased  the 

response  time  and  reduced  the  error 

4.  the  proposed  display  design  was  accepted,  interpreted,  and  used  to  solve 
3D-orientation  tasks  efficiently 
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I.  INTRODUCTION 


Human  performance  in  spatial  orientation  tasks  requires  spatial  awareness  and  the 
skills  of  an  operator  to  transition  from  the  current  spatial  attitude  into  an  acceptable 
spatial  orientation  and  position.  Degraded  spatial  awareness  (SA),  particularly  in  extreme 
flight  situations,  may  lead  to  loss  of  equipment  and  reduced  operational  effectiveness. 
Accidents  and  loss  of  aircraft  due  to  spatial  disorientation  (SD)  are  not  uncommon  in 
manned  aviation.  Loss  of  SA  also  occurs  in  unmanned  aviation,  even  with  operators  on 
the  ground.  Surprisingly,  losses  in  manned  aviation  over  the  last  few  years  are  low  and 
decreasing  but  in  unmanned  aviation,  they  are  high  and  increasing. 

How  to  maintain  and  improve  SA  is  one  of  the  major  issues  in  complex  3D 
environments.  In  particular,  orientation  tasks  in  3D  environments  with  many  degrees  of 
freedom  are  very  difficult  to  accomplish  (Previc  &  Ercoline,  2004).  According  to  Tsang 
&  Vidulich  (2003)  pilots  are  selected  carefully  and  receive  extensive  training  to  maintain 
their  spatial  awareness,  even  in  extreme  fight  situations. 

Different  classes  of  Unmanned  Aerial  Vehicles  (UAV)  systems  operate  in  a  wide 
variety  of  operational  environments,  from  safe,  remote  locations  to  areas  under 
immediate  threat  of  enemy  fire.  This  range  of  operational  environments,  combined  with 
future  plans  for  single-operators  controlling  multiple  UAVs,  leads  to  complex  human- 
system  interfaces. 

However,  the  dynamic  field  of  high-flying  and  Combat  UAV  demands  skilled 
operators  who  are  capable  of  controlling  the  aircraft  remotely.  For  large  UAVs,  pilots  are 
usually  the  first  choice  for  this  task. 

The  cockpit  design  of  UAVs  follows  the  guidelines  of  traditional  flight 
instrumentation  (McCauley  &  Matsangas,  2004).  A  legitimate  question  is  whether  a  new 
prototype  display  design  might  improve  spatial  orientation,  especially  with  an  underlying 
color-coding  feature  when  reaching  an  unusual  attitude.  McCauley  &  Matsangas  (2004) 
also  showed  that  maintaining  spatial  awareness  and  preventing  spatial  disorientation  is  a 
key  factor  when  operating  a  UAV. 
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Since  the  improvement  of  the  operational  utility  of  UAVs,  they  have  increased 
rapidly  in  the  armed  forces  of  many  nations.  They  have  proven  to  be  valuable  in  recent 
military  operations  worldwide.  However,  the  rise  in  UAV  use  has  been  accompanied  by  a 
very  high  mishap  rate  due  to  a  number  of  different  causes  (Williams,  2004). 

The  main  motivation  for  this  thesis  was  to  investigate  the  use  of  an  alternative 
display  design  for  UAV  operators  for  the  purpose  of  improving  spatial  awareness  and 
thereby  reducing  mishaps.  To  evaluate  this  issue,  a  research  questionnaire  was  sent  to 
several  US  Air  Force  units  and  the  information  derived  from  that  source  helped  to  define 
the  research  objectives. 

The  main  goal  of  the  thesis  was  to  determine  whether  a  new  prototype  display 
design,  called  WEBER-Box,  is  a  useful  alternative  or  supplement  to  the  traditional  flight 
instruments  for  determining  and  correcting  for  unusual  attitudes.  The  application 
specifically  investigated  was  operation  of  a  UAV  such  as  Predator.  Another  objective 
was  to  detennine  whether  non-aviator  operators  will  perform  as  well  as  aviators  operating 
a  UAV,  even  in  extreme  flight  situations  or  flight  maneuvers,  when  supported  with 
different  types  of  attitude  indicators.  We  conducted  an  experiment  where  the  participants 
performed  typical  control  tasks  of  a  UAV-operator.  The  experimental  task  required 
participants  to  be  aware  of  their  spatial  orientation  and  to  judge  their  current  attitude 
(pitch  and  roll  orientation).  This  study  investigated  the  potential  of  a  new  display  design, 
the  WEBER-Box,  compared  to  the  traditional  flight  instruments  when  performing  spatial 
orientation  tasks  necessary  to  operate  a  UAV.  The  following  research  questions  were 
addressed: 

•  Will  the  new  display  design  help  the  pilot/operator  to  recognize  the 
current  attitude? 

•  Will  the  new  display  design  help  the  pilot/operator  to  differentiate 
between  a  usual  or  unusual  attitude? 

•  Will  the  new  design  help  to  prevent  misperception  of  an  unusual  attitude 
when  the  UAV  is  not  in  an  unusual  attitude? 

The  core  research  question  to  be  answered  was,  if  the  new  design  supports  the 
recognition  of  the  current  attitude.  An  additional  research  question  was,  if  it  aids  recovery 
when  an  unusual  attitude  exists  and  help  preventing  misperception  of  an  unusual  attitude 
when  the  UAV  is  performing  normally. 
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II.  BACKGROUND  -  COGNITION  IN  AVIATION 


A.  SITUATIONAL  AWARENESS 

SA  is  a  complex  mix  of  many  different  factors.  In  general  it  can  be  defined  ”as 
the  perception,  understanding,  and  the  ability  to  forecast  the  factors  affecting  the  aircraft 
at  any  moment  in  time”  (Moroze  &  Snow,  1999). 

Endsley  (1988)  provided  a  first  established  definition.  According  to  her,  SA  is 
“the  pilot’s  internal  model  of  the  world  around  him  at  any  point  in  time.”  It  is  derived 
from  the  aircraft  instrumentation,  the  out-the -window  view,  and  his  or  her  senses. 
Individual  capabilities,  training,  experience,  objectives,  and  the  ability  to  respond  to  task 
workload  moderate  the  quality  of  the  operator’s  SA.  She  expanded  in  1995  her  definition 
by  addressing  for  the  first  time  three  different  levels.  SA  provides  “the  primary  basis  for 
subsequent  decision  making  and  perfonnance  in  the  operation  of  complex,  dynamic 
systems...”  At  its  lowest  level  the  operator  needs  to  perceive  relevant  information,  next 
he  has  to  integrate  the  data  in  conjunction  with  task  goals,  and  at  its  highest  level,  predict 
future  events  and  system  states  based  on  this  understanding.  Vidulich  (1995)  describes  it 
succinctly  as  referring  to  the  pilot’s  cognitive  understanding  of  the  current  situation  and 
its  implications.  According  to  Jaslow  (1998),  SA  is  the  knowledge  of  the  aircraft’s 
location  in  space  (position  and  attitude),  awareness  of  the  environment  (terrain  geography 
and  features,  weather-related  conditions  such  as  turbulence  and  icing),  understanding 
communication  (within  the  cockpit  and  via  the  air  traffic  controller),  correct  reading  of 
the  instruments,  and  knowledge  of  malfunctioning  controls  and /  or  instruments.  This 
definition  provides  the  wide  approach  of  SA.  It  also  indicates  that  spatial  orientation,  as 
shown  in  a  succeeding  chapter,  is  only  one  part  of  SA,  because  it  only  describes  the 
knowledge  of  the  aircraft’s  orientation  in  space  (Gawron,  2004). 

Endsley  (2000)  described  this  information  coherence  as  an  information  gap  (see 
Figure  1). 
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Figure  1.  The  information  gap  (Endsley,  2000) 


In  addition,  Endsley  (1999  and  2000)  discusses  three  different  levels  of  SA: 


Level  1:  Perception 


Level  2:  Comprehension 


Level  3:  Projection 


The  first  step  “is  to  perceive  the  status,  attributes,  and 
dynamics  of  relevant  information  in  the  environment” 
(Endsley,  1999),  which  is  fundamental  in  a  current 
situation.  Without  the  perception  of  the  most 
important  information,  the  option  of  building  an 
incorrect  picture  of  the  situation  increases. 

Comprehension  expands  the  first  level  by  under¬ 
standing  the  importance  of  those  elements. 
Combining,  interpreting,  storing,  and  retaining  these 
information  is  gaining  importance. 

Deriving  lessons  and  consequences  from  the  former 
levels  enables  the  operator  to  project  future  events 
and  actions. 
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The  interaction,  not  only  of  the  three  levels  of  SA,  but  also  the  whole  complex  of 
SA  can  be  illustrated  with  Figure  2. 


Figure  2.  Model  of  SA  in  dynamic  decision  making  (Endsley,  2000) 


B.  SPATIAL  AWARENESS 

To  understand  the  term  “spatial  awareness”  and  to  put  it  into  the  right  context,  it 
is  important  to  know  that  it  is  only  one  component  of  SA.  The  other  components  of  SA 
are  system  awareness  and  task  awareness.  Each  of  these  components  has  real-world 
implications,  spatial  awareness  for  example  for  instrument  displays. 

Therefore  spatial  awareness  is  the  understanding  of  the  location,  in  3D-space,  of 
particular  objects  within  a  given  environment.  Consequently,  spatial  awareness  is 
necessary  in  order  to  know,  for  example,  the  relative  location  of  enemy  targets,  friendly 
or  neutral  forces.  Increasing  one’s  conceptual  understanding  of  a  particular  3D-space, 
also  increases  one’s  spatial  awareness.  It  is  important  to  gain  a  mental  picture  of  the 
current  situation  because  with  this  ability  a  pilot  or  operator  is  more  able  to  predict  the 
future  situation  and  thus  be  able  to  operate  the  aircraft  proactively  (Wickens,  2002). 
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Wickens  (2002b)  also  describes  six  different  variables  that  are  important  for  the 
mental  picture  of  a  pilot: 

•  pitch 

•  roll 

•  yaw 

•  altitude 

•  deviation  from  a  flight  path 

•  position  along  a  flight  path 

Thus  flight  instruments  which  are  displaying  information  about  these  variables  in 
an  appropriate  manner  contribute  significantly  to  the  spatial  awareness  of  a  pilot  or 
operator  of  a  UAV. 


C.  SPATIAL  ORIENTATION  /  DISORIENTATION 

In  general,  humans  are  adapted  to  maintain  spatial  orientation  on  the  ground. 
Since  they  are  not  designed  for  the  three-dimensional  environment  of  flying,  which  is 
unfamiliar  to  their  body  systems,  sensory  conflicts  and  illusions  are  almost  inevitable. 
Therefore,  spatial  orientation  is  sometimes  difficult,  even  impossible  to  achieve 
(Antunano,  2003).  Three  sensory  systems  (see  Figure  3  and  Figure  4)  provide  the 
information  necessary  to  maintain  the  equilibrium  and  determine  where  we  are  and  how 
we  are  orientated: 

•  Visual  system:  The  eyes,  which  sense  position  and  movement  based  on 
optical  infonnation. 

•  Vestibular  system:  Organs  of  balance  found  in  the  inner  ear  that  sense 
linear  and  angular  acceleration  and  support  posture  and  locomotion. 

•  Somatosensory  system:  Nerves  in  the  skin,  muscles,  and  joints, 

which,  along  with  hearing,  sense  position  and  movement  based  on  gravity, 
feeling,  and  sound. 
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Figure  3.  Spatial  orientation  (Antunano,  2000) 


Figure  4.  Spatial  orientation  (Antunano,  2003) 

Spatial  orientation  in  flight  is  difficult  to  achieve  because  the  three  different 
sensory  systems  vary  in  magnitude,  direction,  and  frequency.  Any  differences  or 
discrepancies  result  in  a  sensory  mismatch  that  can  produce  illusions  and  lead  to  spatial 
disorientation.  Good  spatial  orientation  relies  on  the  effective  perception,  integration  and 
interpretation  of  their  sensory  information  (Antunano,  2003).  Thus  one  major  issue  is  the 
question  of  appropriate  countenneasures.  The  most  common  one  is  to  “trust  your 
instruments.”  On  the  other  hand,  flying  by  instruments  increases  demands  on  mental 
resources,  task  load,  and  disorientation  stress  (Braithwaite,  Dumford,  DeRoche,  Alvarez, 
Jones,  Flidgon  et  ah,  1997). 
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SD  is  a  failure  “to  sense  correctly  the  position,  motion,  or  attitude  of  the  aircraft 
or  of  him /  herself  within  the  fixed  coordinate  system  provided  by  the  surface  of  the  earth 
and  the  gravitational  vertical”  (Benson,  1988).  In  addition,  errors  in  perception  by  the 
aviator  of  his  position,  motion,  or  attitude  with  respect  to  his  aircraft,  or  of  his  own 
aircraft  relative  to  other  aircraft,  may  also  be  embraced  within  a  broader  definition  of  SD. 
Any  condition  that  deprives  the  pilot  of  natural,  visual  references  to  maintain  orientation 
can  rapidly  cause  SD.  Regardless  of  a  pilot’s  experience  or  proficiency,  sensory  illusions 
can  lead  to  differences  between  instrument  indications  and  what  the  pilot  feels  the  aircraft 
is  doing. 

SD  in  flight  operations  has  been  a  common  problem  since  the  early  days  of 
aviation.  Accidents  continue  to  occur  despite  improvements  in  the  display  of  information 
to  the  pilot.  Even  with  improvements  of  manufacturing  technologies,  quality  control,  and 
aircraft  maintenance,  accidents  still  occur.  Reasons  for  this  fact  are  related  for  example  to 
new  technologies  such  as  night  vision  goggles.  Flight  operations  can  now  be  executed  in 
environments  that  have  not  been  possible  before  (Benson,  1988). 

Between  1990  and  1999,  the  United  States  Air  Force  (USAF)  experienced  36  SD- 
related  Class  A  mishaps.  These  have  cost  a  total  of  $557M  and  even  more  important,  the 
loss  of  44  aircrew.  These  economic  consequences  of  SD  are  enormous,  both  in  cost  of 
lost  aircraft,  lost  aircrew,  and  cost  of  training  new  aircrew  (Heinle  &  Ercoline,  2003).  For 
example,  in  the  period  1992  -  2000,  SD  was  the  cause  of  20.2%  of  USAF  Class  A 
mishaps.  Respectively  US  Army  with  27%  and  US  Navy  with  26%  rate  in  an  equivalent 
period.  But  on  the  other  hand,  in  a  period  of  1972  -  2000  USAF  Class  A  mishaps  have 
decreased  in  an  overall  rate  from  2.5  accidents  per  100.000  flying  hours  in  1972  to  a  rate 
of  1.0  in  2002.  Despite  this  decrease,  the  rate  of  SD  has  only  fallen  from  0.5  to  0.25  and 
has  remained  constant  in  the  last  15  years  (Benson,  2000).  Therefore,  in  general,  SD  is 
still  the  single  most  common  cause  of  human-related  aircraft  accidents  (Heinle  & 
Ercoline,  2003). 

The  literature  distinguishes  between  three  different  types  of  SD.  About  80%  of  all 
Class  A  SD  accidents  are  of  type  I  and  unrecognized.  Pilots  do  not  consciously  perceive 
any  manifestation  of  SD  and  most  often  it  occurs  when  he/  she  breaks  a  cross-check. 
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Most  likely  it  also  leads  into  a  controlled  flight  into  terrain.  20%  of  all  SD  Class  A 
accidents  are  form  type  II.  The  pilot  consciously  perceives  a  manifestation  of  SD  but  may 
not  attribute  it  to  SD.  Often,  he  suspects  an  instrument  malfunction.  The  most  well- 
known  example  is  the  Graveyard  spin  (see  Figure  5).  In  this  case  the  pilot  realizes  a 
conflict  between  the  information  displayed  by  the  instruments  and  the  feeling  generated 
by  the  inner  ear.  The  pilot  has  to  decide  which  sensory  system  to  believe.  If  he/  she  trust 
his/  her  inner  ear,  the  aircraft  may  spin  all  the  way  to  ground  impact.  The  last  type,  type 
III,  of  SD,  also  called  incapacitating  SD,  is  the  least  common,  known  and  understood. 
Few  written  reports  exist  at  all,  but  it  is  known  through  experience  and  pilot  reports.  An 
example  is  named  “giant  hand  illusion”  (see  Figure  6).  Sometimes  pilots  feel  that  they  are 
unable  to  move  the  controls  in  one  direction,  although  they  are  capable  of  moving  the 
controls  in  any  other  direction.  A  suggested  countermeasure  is  to  remove  the  hands  from 
the  controls  and  try  it  again  by  only  using  fingers  without  arm  movements.  Even  a 
technical  inspection  after  the  flight  showed  no  evidence  of  a  technical  malfunction  and  it 
is  not  possible  to  duplicate  the  problem  (Heinle  &  Ercoline,  2002). 


Figure  5.  Graveyard  Spin  (Heinle  &  Ercoline,  2002) 
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Figure  6.  Giant  Hand  Illusion  (Heinle  &  Ercoline,  2002) 

Each  type  of  SD  affects  the  pilot  in  a  different  way,  and  each  should  be 
thoroughly  understood  by  the  pilot  before  he  or  she  experiences  them  in  flight.  Although 
the  phenomenon  of  SD  has  been  described  and  documented  by  many,  both  researcher  and 
aircrew,  since  the  earliest  days  of  aviation,  a  complete  understanding  of  the  complex 
mechanisms  and  interactions  has  remained  elusive. 

According  to  Benson,  2002,  the  human  factors  and  aero-medical  community  also 
has  to  be  aware  of  SD  in  virtual  reality  and  in  operating  UAVs.  He  defines  SD  as  a 
misperception  of  the  key  elements  in  the  flight  environment.  These  key  elements  may 
include  navigation,  weather,  tactics,  nature  of  threats,  aircraft  systems,  and  spatial 
orientation.  The  task  of  piloting  a  UAV  may  be  sensitive  to  SD,  especially  in  cases  of 
highly  maneuverable  vehicles,  such  as  Combat-UAV  where  information  is  provided  by 
optical  and  other  sensors,  and  in  extreme  environmental  conditions,  like  high  wind 
speeds.  In  these  cases,  he  concludes,  there  is  a  considerable  potential  for  pilot 
disorientation  and  loss  of  control.  Several  factors,  like  restricted  field  of  view,  and  lags  in 
the  display  of  information  can  be  causes  of  mishaps.  In  addition,  there  is  also  the  lack  of 


10 


motion  stimuli  to  the  operator’s  body,  in  contrast  to  manned  aviation.  Unlike  in 
conventional  aviation,  SD  is  widely  underestimated  for  unmanned  aviation.  Since 
unmanned  aviation  lacks  the  direct  physical  perception  and  feedback,  we  expect  that 
improved  display  concepts  will  contribute  to  a  better  spatial  awareness  and,  hence,  will 
decrease  the  potential  for  SD.  A  10-year  cross  sectional  review  of  human  factors  in  UAV 
mishaps  within  the  U.  S.  Department  of  Defense  (DoD)  supports  Benson’s  position. 
Interestingly,  the  outcome  of  this  DoD-review  is  that  the  misperception  error  was  present 
in  5%  of  all  UAV  mishaps. 
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III.  COCKPIT  AND  DISPLAY  DESIGN 


A.  GENERAL  PRINCIPLES  DISPLAY  DESIGN 

The  design  of  flight  instruments  and  the  cockpit  panel  have  a  long  history, 
beginning  with  Elmer  Sperry  and  the  Airborne  Gyro  (Hughes,  1971).  Even  in  the  earliest 
powered  flights,  keeping  the  simple  planes  steady  was  difficult.  The  addition  of 
gyroscopes  in  1914  brought  new  instruments  to  the  cockpit,  helping  the  pilot  to  maintain 
steady  control.  Gyroscopes  and  their  derivatives  remain  an  important  part  of  flight 
technology.  Basically  they  have  not  changed  since  the  beginning  of  aviation.  Moreover, 
they  have  been,  and  still  are,  the  pilot’s  main  source  of  information.  They  are  his  tool  for 
flying  in  an  usual  attitude,  recovering  from  an  unusual  attitude,  and  for  preventing 
mishaps. 

Traditional  cockpit  design  and  flight  instruments  have  almost  always  been  arrayed 
with  the  most  important  instruments  in  the  shape  of  a  T  (see  Figure  7).  This  configuration 
can  be  found  in  a  wide  variety  of  aircraft  from  the  beginning  of  aviation  to  the  latest 
models  of  aircraft. 


Figure  7.  Basic  layout  of  flight  instruments  (Anderson,  2002) 
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The  center  spot  is  always  reserved  for  the  artificial  horizon  or  attitude  indicator, 
which  is  not  only  one  of  the  oldest  instruments  but  also  one  of  the  most  important.  It 
provides  information  to  the  pilot  about  his  current  situation  in  space  and  therefore 
displays  the  flight  attitude  relative  to  the  earth  (see  Figure  8).  Pitch  attitude  is  indicated 
by  up  and  down  motion  of  the  sphere  with  respect  to  horizontal  reference.  Roll  attitude  is 
indicated  by  rotational  motion  of  roll  pointer  with  respect  to  the  fixed  roll  scale  located  at 
top  of  the  display. 


Figure  8.  Traditional  attitude  indicator  (screenshot  of 
X-Plane  ™  flight  simulation) 

The  evolution  of  aviation  from  the  beginning  with  the  Wright  brothers’  to  modern 
aviation  has  brought  some  significant  changes  in  the  capabilities  of  the  aircraft  (Tsang  & 
Vidulich,  2003).  Besides  the  instruments,  which  basically  remained  the  same,  other 
factors  like  speed  of  the  aircraft,  especially  for  fighter  aircraft,  have  changed.  The 
demands  on  the  pilots  have  thus  increased,  the  workload  is  higher,  and  the  task  is  more 
complex  now. 

With  the  development  of  new  technologies  in  flight  instrument  design,  there  has 
been  a  shift  away  from  individual  displays  like  altimeter,  airspeed  indicator,  or  tum-and- 
bank  indicator,  towards  integrated  displays  in  which  most  of  the  information  comes  from 
a  common  source  like  a  heads-up  display  (FIUD)  or  helmet-mounted  display  (HMD). 
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Independent  of  the  state-of-the-art  display  design,  three  basic  issues  can  be 
“applied  to  the  design  and  modification  of  primary  flight  and  navigation  instruments  in 
the  context  of  situation  awareness  assessment”  (Andre,  Wickens  &  Moonnan,  1991): 

•  Display  perspective:  two-  vs.  three-dimensional 

•  Frame  of  reference:  inside-out  vs.  outside-in 

•  Visual  display  monochrome  vs.  color 


Many  displays,  like  traffic,  weather,  or  navigational  displays,  are  two-dimensional 
with  a  “God’s  eye”  plan  view  of  the  earth.  On  the  other  hand,  flying  an  aircraft  is  much 
more  related  to  a  three-dimensional  task.  In  this  context,  three-dimensional  means  the  use 
of  “perceptual  depth  or  distance  cues  to  create  a  three-dimensional  image  (Wickens, 
2003).” 


An  important  feature  of  3-D  displays  is  the  frame  of  reference  or  the  viewpoint 
from  which  a  selected  part  of  interest  is  presented  (Wickens,  2003).  Thus  the  point  of 
view  is  important  for  a  comparison  between  flight  instruments.  Previc  &  Ercoline,  1999, 
describe  the  fundamental  differences  between  the  first  two  types  of  attitude  presentation 
for  an  aircraft  in  the  following  way: 

•  Egocentric  or  Immersed  view  point:  This  type  is  also  known  as  “inside- 
out”  or  “moving  horizon  ”  or  “moving  outside  world”.  The  outside  world 
is  presented  from  the  inside  perspective  of  an  aircraft  to  the  outside  world 
(see  Figure  9,  right).  The  human  is  fixed  in  space  and  the  world  around 
him  moves. 

•  Exocentric  or  tethered  view  point:  This  type  is  also  known  as  “outside-  in” 
or  “moving  aircraft”.  The  perspective  is  from  the  outside  world  into  the 
aircraft.  This  type  of  attitude  presentation  is  rarely  used  for  UAVs  (see 
Figure  9,  left).  The  world  is  fixed  and  the  human  moves. 

•  Coplaner  view  point:  It  shows  a  top-down  lateral  view  and  a  vertical 
profile  view,  which  can  also  have  a  side  view  or  a  back  view  (Wickens, 
2003).  Therefore  it  is  a  2-D  display,  contrary  to  egocentric  or  exocentric 
that  are  3-D  display  designs. 
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Moving  Aircraft 


Moving  Outside  World 


Figure  9.  Fundamental  differences  of  attitude  representation 
for  an  aircraft  (Previc  &  Ercoline,  1999) 

According  to  Clay,  1993,  many  aircraft  accidents  are  due  to  SD  problems  of  the 
pilot  and  to  an  additional  wrong  interpretation  of  the  flight  instruments.  One  of  the  major 
challenges  for  flight  instrument  display  is  the  tremendous  data  flow  and  information 
available  from  all  of  the  instruments.  Additionally,  the  priority  of  recalling  the 
information  changes  with  every  task.  In  addition,  flight  instruments  are  normally  not  very 
intuitive  (Previc  &  Ercoline,  2004).  On  the  other  side,  multifunctional  instruments  and 
the  “glass  cockpit”  have  been  developed.  This  approach  may  lead  to  an  overload  of  the 
pilot/  operator  because  more  information  is  available,  the  number  and  complexity  of 
display  pages  has  increased.  Furthermore,  with  an  increasing  number  of  warning  signals 
like  audio  or  visual,  status  displays,  flight  path  controls,  only  to  mention  a  few,  the 
human  processing  ability  may  be  exceeded  (Mejdal  &  McCauley,  2001).  As  a  result,  the 
requirements  for  developing  new  display  designs  or  improving  existing  display  designs 
become  important.  This  display  design  challenge  is  taken  in  a  new  direction  when  UAVs 
are  taken  into  account.  For  this  purpose,  display  designs  have  to  be  less  abstract  and 
intuitively  understandable  for  an  operator  on  the  ground. 
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The  visual  impact  of  color  is  strong  and  has  been  used  in  simple  display 
applications  since  before  the  Second  World  War.  The  advantage  of  color  for  flight 
instruments  is  enhancing  the  presentation  of  information  and  gaining  user  acceptance. 
Color  has  proven  effective  when  a  lot  of  information  must  be  presented.  Two  different 
applications  of  color-coding  are  widely  used.  First,  color  can  be  used  to  signify  quantity, 
that  is,  to  categorize  data.  The  airspeed  indicator  is  an  example  for  this  type  of 
application.  Secondly  “qualitative  color  displays  classify  data  by  type  or  condition  and 
are  often  used  to  convey  status  information”  (Stokes  &  Wickens,  1988).  For  this 
application,  color  can  be  used  to  facilitate  selective  attention.  It  can  act  as  an  attention 
cue  for  the  pilot  who  will  never  be  able  to  monitor  all  his  displays  at  the  same  time  with 
the  same  level  of  attention.  Typically  colors  for  danger,  caution,  or  advisory  information, 
are  red,  amber,  or  green,  respectively  (Stokes  &  Wickens,  1988). 


B.  INSIDE-OUT  VERSUS  OUTSIDE-IN  DISPLAY  DESIGN 

There  is  a  general  discussion  between  researchers  as  to  which  type  of  design  of 
attitude  presentation  human  beings  prefer.  One  very  early  approach  to  utilizing  an 
exocentric  instrument  design  was  a  study  by  Hennessy,  Lintern  &  Collyer,  1981,  that 
used  an  exocentric  view  on  an  aircraft  model  to  facilitate  the  flight  training  of  pilots.  Four 
different  visual  displays  were  evaluated  for  their  effectiveness  in  the  acquisition  of  flight 
tasks  in  a  flight  simulator.  The  experimental  displays  were, 

(a)  wide-field-of-view  ( 1 60°)  with  horizon  and  checkerboard  ground  plane 

(b)  narrow-field-of-view  (48°)  with  horizon  and  checkerboard  ground  plane 

(c)  an  outside-in  view  of  an  airplane,  also  referred  by  the  term 
“unconventional  visual  display” 

(d)  a  display  that  consisted  only  of  traditional  flight  instruments. 

The  participants  were  flight  students  who  had  to  learn  basic  flight  tasks  in  a 
simulator.  The  experiment  was  conducted  in  form  of  20  trials  (4  per  display  type).  The 
researchers  were  particularly  interested  in  the  transfer  of  training  in  terms  of  how 
accurate  a  flight  student  could  perform  certain  simple  flight  maneuvers.  The  key  finding 
was  that  the  students  were  able  to  leam  flight  maneuvers  faster  when  using  the  outside-in 
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display.  Furthermore,  the  results  showed  that  it  is  advantageous  to  have  an  external  point 
of  view  for  perceiving  and  understanding  flight  behaviors. 

Previc  &  Ercoline  (1999)  presented  additional  findings  on  outside-in  flight 
displays,  claiming  that  they  are  superior  at  preventing  roll-reversal  errors  during  nonnal 
flight  and  in  recovering  from  extreme  flight  situations.  They  state  that  “Although  outside- 
in  attitude  displays  have  been,  and  continue  to  be,  flown  throughout  the  world,  the 
majority  of  military  and  civilian  aircraft  have  failed  to  adopt  this  format”  (Previc  & 
Ercoline,  1999).  One  of  their  conclusions  is  that  even  experienced  pilots  do  not  show  a 
clear  advantage  with  an  inside-out  attitude  display  with  which  they  have  trained 
extensively.  In  contrast,  pilots  trained  on  an  outside-in  format  show  overwhelming 
advantage  in  recovering  from  unusual  attitude. 

Self,  Breun,  Feldt,  Perry  &  Ercoline,  2002,  conducted  a  study  for  students  in  an 
introductory  flight  course  at  the  U.S.  Air  Force  Academy.  They  were  tested  on  three 
different  display  symbologies: 

(a)  moving  horizon  symbology  (egocentric  viewpoint) 

(b)  moving  plane  symbology  (exocentric  viewpoint) 

(c)  arc-segment  attitude  reference  symbology. 

The  researchers  evaluated  the  time  a  student  needs  to  give  a  roll-input  in  order  to 
correct  a  certain  spatial  orientation  of  the  aircraft  as  their  main  parameter  of  interest.  This 
study  found  that  the  exocentric  “outside-in”  attitude  display  and  the  arc-segment  attitude 
reference  symbology  were  superior  in  reducing  the  time  of  roll-inputs  in  normal  flights. 

Tsang  &  Vidulich,  2003b,  confirmed  the  former  findings  that  pilots  prefer  an 
exocentric  or  outside-in  view  of  the  world. 

C.  PICTORIAL  DISPLAY  DESIGN 

Research  on  pictorial  display  design  began  in  early  1990s  when  National 
Aeronautics  and  Space  Administration  (NASA)  conducted  a  series  of  experiments  in 
order  to  determine  if  landing  and  taxing  approaches  of  commercial  aircraft  and  Space 
Shuttles  may  benefit  from  an  external  view  of  the  own  aircraft.  Additionally  Parrish, 
Busquets,  Williams  &  Nold,  (1994)  conducted  an  experiment  to  evaluate  certain  display 
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types,  including  a  pictorial  display  design,  which  is  also  called  “pathway  in  the  sky”  or 
“highway  in  the  sky”.  The  objective  was  to  evaluate  and  compare  spatial  awareness 
components  of  pilots  using  various  display  concepts.  These  four  are: 

(a)  conventional  electronic  flight  information  systems  (EFIS) 

(b)  EFIS  and  speech  commands 

(c)  pictorial  “pathway  in  the  sky”  display  with  70°  field-of  view  and 

(d)  a  pictorial  “pathway  in  the  sky”  display  with  40°  field-of  view. 

The  pictorial  display  was  a  computer  rendered  graphic  of  an  out-the-window 
scene  with  a  flight  path  that  was  displayed  as  green  goalposts  with  width  and  height  in 
correspondence  with  the  input  from  the  electronic  landing  system.  The  results  showed 
greatly  improved  subjective  rankings  of  the  pictorial  design  in  terms  of  reduced 
workload,  effort  to  monitor  traffic  and  overall  ranking.  The  objective  measurements 
reveal  a  significant  decrease  of  maneuver  errors  in  all  monitored  categories. 

This  result  is  also  supported  by  Ercoline,  DeVilbiss  &  Evans  (2004)  because  the 
pictorial  design  benefits  the  pilot’s  spatial  orientation.  The  configuration  of  the  display 
appears  more  like  the  real  world  because  a  symbolic  path  is  used  to  represent  “the  desired 
vertical  and  horizontal  trajectory  relative  to  Earth’s  surface  as  opposed  to  command 
steering  bars  or  a  single  command  steering  cue  (Ercoline  et  ah,  2004).”  In  addition,  the 
pictorial  display  is  appreciated  by  novices  and  experienced  pilots. 


D.  THE  WEBER-BOX 

The  WEBER-Box  is  a  design  concept  for  flight  instruments  that  emphasizes  a 
three-dimensional  exocentric  view  of  a  virtual  aircraft  (Weber,  2006).  He  proposed  in  his 
thesis  a  new  human-centric  design  approach  for  a  3D-flight  instrument  based  on  “avatar” 
concepts.  Weber  (2006)  combined  an  exocentric  design  with  pictorial  metaphors  of  the 
current  spatial  orientation.  By  animating  the  entire  scene,  a  story  is  being  told  about  what 
happens  during  a  flight  maneuver.  In  this  context,  it  is  a  “miniature  abstract  virtual 
world”  that  represents  the  actual  spatial  situation  of  the  aircraft.  The  avatar  “lives”  in  this 
virtual  world.  The  display  representing  the  aircraft  is  a  3D  wire-frame  model  that  hovers 
inside  an  abstract  box  and  replicates  all  the  basic  motions  of  the  real  aircraft.  The  frame 
of  reference  is  a  coordinate  system,  consisting  of  X-,  Y-,  and  Z-axes.  Altitude  is 
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represented  by  a  3D  bar  along  the  Y-  axis.  The  aircraft’s  movement  relative  to  the  ground 
is  displayed  by  moving  objects  along  the  X-/Z-  plane  (see  Figure  10  and  11). 

Weber  (2006)  conducted  two  experiments,  which  found  strong  evidence  for 
improvement  of  spatial  awareness  and  pilot’s  performance  in  extreme  flight  situations. 
His  findings  were  that  the  participants  were  able  to  judge  the  spatial  orientation  of  their 
own  aircraft  about  three  times  faster,  compared  to  traditional  flight  instruments.  The  time 
needed  to  recover  the  aircraft  from  unusual  attitudes  improved  by  about  70%. 
Furthermore,  he  found  no  statistically  significant  differences  in  terms  of  performance 
improvement  between  pilots  and  non-pilots.  Thus,  operators  of  any  experience  level 
might  benefit  from  the  design  of  the  “WEBER-Box”. 


Figure  10.  Screen  shot  of  the  WEBER-Box 
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Figure  1 1 .  Screen  shot  of  the  WEBER-Box 


Studies  show  that  maintaining  spatial  awareness  and  preventing  SD  is  becoming 
more  and  more  an  issue  when  operating  UAVs.  Current  UAV  controls  rely  on  traditional 
flight  instruments,  which  are  inherently  difficult  to  understand  and  demand  a  high  degree 
of  training  (McCauley  &  Matsangas,  2004).  On  the  other  hand,  human-centric  display 
design  is  likely  to  substitute  or  supplement  traditional  flight  instruments  in  the  future.  The 
outside-in  display  formats,  combined  with  pictorial  visualization  seems  to  be  promising 
to  support  and  improve  spatial  awareness  for  operators  of  UAVs.  The  design  concept  of 
the  WEBER-Box  combines  those  display  principles. 
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IV.  UNMANNED  AERIAL  VEHICLES 


A.  HISTORY  AND  OVERVIEW 

Traditional  aircraft  have  an  important  disadvantage  or  weakness.  They  have  to  be 
piloted  by  humans.  Therefore  there  are  negative  consequences  which  have  to  be 
considered.  In  case  of  a  crash,  you  not  only  loose  equipment  worth  millions  of  dollars  but 
also  may  loose  the  pilot.  Other  negative  outcomes  in  military  operations  include  the  pilot 
and  aircrew  being  taken  as  prisoners  of  war  or  hostages,  which  could  have  a  serious 
impact  on  public  opinion  or  the  way  a  conflict  has  to  be  fought. 

Thus,  the  idea  of  developing  uninhabited  or  unmanned  aerial  vehicles  arose.  Since 
1964  the  United  States  Military  has  spent  billions  of  dollars  researching,  developing  and 
constructing  countless  types  of  UAVs.  The  United  States  Air  Force  has  used  UAVs  for 
reconnaissance  and  surveillance  at  very  long  ranges  up  to  3,000  miles  away.  The  core 
idea  is  to  provide  the  warfighter  with  intelligence  information  and  a  precise  delivery 
mechanism  for  smart  area  weapons,  non-lethal  payloads,  as  well  as  other  cargo.  Recently 
UAVs  have  also  been  established  by  the  US  Army  and  Marine  Corps  as  tactical  assets  on 
the  battlefield.  Figure  12  provides  a  good  impression  of  the  actual  size  of  a  UAV  Predator 
and  Global  Hawk  compared  to  a  Boeing  737  and  therefore  how  far  the  development  for 
large  UAVs  has  already  gone. 


Figure  12.  Sizes  of  UAV  Predator  and  Global  Hawk  compared  to  a  Boeing  737 

(Picture  by  Northrop  Grumman) 
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On  the  other  hand,  the  latest  and  additional  development  points  into  the  direction 
of  Micro  UAVs  which  can  be  launched,  for  example,  in  urban  warfare. 


UAV  Evolution  -  Where  are  we? 
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Figure  13.  UAV  Evolution  -  Where  are  we?  (AUVSI,  2001) 


There  is  not  only  the  question  “Where  are  we?”  but  even  more  interesting  is  the 
question  “Where  are  we  going?”  Currently  under  development  is  the  Automated  Aerial 
Refueling  (AAR)  that  could  demonstrate  for  the  first  time  a  UAV’s  ability  to 
autonomously  maintain  a  permanent  refueling  station  behind  a  tanker  aircraft.  This 
extends  the  range  of  a  UAV  tremendously  because  now  only  the  crew  in  the  GGS  has  to 
be  exchanged  but  the  UAV  can  continue  its  mission. 

Even  the  goals  for  developing  and  building  future  UAVs  are  detennined,  there  are 
still  concerns  (see  Table  1)  that  have  to  be  taken  care  of  (Thompson,  2000). 
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Datallnk 

Communica¬ 

tions 

1.  Loss  of  control  due  to  enemy  jamming  or  signal  manipulation 

2.  Long  connectivity  lapses  due  to  distance,  satellite  location,  or  friendly  mutual  interference 

3.  Limited  amount  of  frequency  bandwidths  to  accommodate  large  numbers  of  secure  links 
for  multiple  UCAV  operations 

Air  Refueling 

1.  Transoceanic  deployment  distances  and  communications 

2.  Risk  to  KC-135  or  KC-10  high-value  assets 

3.  Tanker  joinup  and  multiaircraft  air  refueling 

Operator 

Situational 

Awareness 

1.  Number  of  aircraft  per  operator  or  operators  per  aircraft 

2.  Air  traffic  control  (ATC)  and  enemy  airspace  deconfliction  from  other  aircraft 

3.  Threat  reactions  for  visual  antiaircraft  artillery,  infrared  surface-to-air  missiles,  or  enemy 
aircraft 

Emergencies 

1.  Less  capability  to  rapidly  assess  and  correct  aircraft  problems 

2.  Unable  to  see  damage,  feel  small  vibrations,  or  smell  smoke 

3.  UAV-capable  alternate  airfield  recovery  due  to  fuel  or  weather 

Table  1.  Future  UCAV  concerns  (Thompson,  2000) 


B.  ACCIDENTS 

High  mishap  rates  are  often  cited  as  a  deterrent  to  a  wider  adoption  of  UAVs  into 
future  force  structure.  Over  the  last  five  decades,  investments  in  manned  aircraft 
reliability  have  been  made  to  drive  equipment  failures  to  near  zero.  This  progress  implies 
that  a  considerable  reduction  in  UAV  accident  rates  due  to  technical  failure  also  can  be 
obtained  with  reasonable  investments.  On  the  other  hand,  it  is  also  likely  that  the  human 
in  the  loop  contributes  as  a  main  source  of  failure  and  high  mishap  rates  for  both  manned 
and  unmanned  aviation.  A  significant  contributor  to  UAV  mishaps  is  the  experience  level 
of  UAV  operators  and  maintainers. 

Williams  (2004)  classified  UAV  accident  data  into  the  categories  of  human 
factors,  aircraft,  and  unknown.  He  analyzed  explicitly  the  data  of  the  following  UAVs: 
Hunter,  Shadow,  Pioneer,  Predator,  and  Global  Hawk,  which  are  currently  in  use  in  the 
American  Forces.  His  findings  are  that  the  involvement  of  human  factors  was  high,  in  a 
range  from  21%  to  68%,  depending  on  the  type  of  UAV.  Various  human  factors  issues 
have  been  observed,  including  handing  over  a  UAV  from  one  pilot  to  another,  launch  and 
recovery,  display  issues,  and  disorientation. 

DoD  (2005)  compared  in  its  roadmap  2005  -  2030  the  UAV  mishap  rate  with  the 
mishap  rate  of  the  F-16  (see  Figure  14)  of  class  A  mishaps.  These  are  the  mishaps 
resulting  in  the  loss  of  the  aircraft,  human  life,  or  causing  $1,000,000  in  damage.  It 
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illustrates  the  mishap  rate  after  similar  numbers  of  flying  hours  have  been  accumulated. 
This  Figure  shows  significantly  that  the  mishap  rate  of  today’s  UAV  is  comparable  with 
the  mishap  rate  of  early  manned  military  aircraft. 


Cumulative  Flight  Hours 


Figure  14.  Mishap  rate  comparison  (DoD,  2005) 

An  additional  approach  is  the  vulnerability  of  UAVs  in  combat  situations.  Many 
of  the  current  UAVs  are  vulnerable  to  different  air  defense  systems,  which  of  course 
depends  on  the  altitude  and  the  speed  of  the  UAV.  Certainly  this  applies  to  tactical  UAVs 
up  to  Medium- Altitude,  Long-Endurance  (MALE)  UAVs  like  the  Predator.  Anti-aircraft 
artillery  (AAA),  shoulder-fired  man  portable  systems  (MANPADS),  and  radar  directed 
low  medium  and  high  altitude  surface  to  air  missiles  can  all  be  quite  lethal  to  UAVs 
within  their  range.  Currently  the  concept  of  operations  calls  for  air  superiority  and  the 
elimination  of  the  air  defense  threats  before  a  UAV  is  sent  into  this  area.  In  the  future,  the 
concept  of  operations  could  be  extended  if  active  countermeasures  were  installed  on 
UAVs,  which  would  enable  their  mission  to  be  extended  to  surveillance  prior  to  a 
conflict. 
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C.  UAV  SYSTEM  EXAMPLE:  PREDATOR 

Smaller  in  size  compared  to  its  larger  brother  Global  Hawk,  the  27-foot-long 
UAV  Predator  is  much  cheaper,  $4.5  million  each.  As  a  result,  it  may  be  worth  the  risk  of 
being  sighted  and  shot-down  to  fly  a  Predator  at  relatively  low  altitudes  (25,000  feet  and 
below). 

Predators  were  first  deployed  for  reconnaissance  and  surveillance  operations  by 
the  US  military  during  the  1995  civil  war  in  Bosnia.  They  are  now  far  more  extensively 
used  by  the  USAF  in  the  operations  such  as  Afghanistan. 


The  Predator  is  a  medium-altitude,  long-endurance  remotely  piloted  aircraft.  Its 
primary  mission  is  interdiction  and  conducting  armed  reconnaissance  against  critical, 
perishable  targets.  When  it  is  not  actively  pursuing  its  primary  mission,  it  acts  as  the  Joint 
Forces  Air  Component  Commander-owned  theater  asset  for  reconnaissance,  surveillance 
and  target  acquisition  in  support  of  the  Joint  Forces  Commander. 


The  basic  crew  is  one  air  vehicle  operator  (AVO)  or  pilot  and  two  sensor 
operators.  They  fly  the  aircraft  from  inside  the  ground  control  station  via  a  line-of-sight 
data  link  or  a  satellite  data  link  for  beyond  line-of-sight  flight  (see  Figure  15). 


Figure  15.  Data  and  satellite  link  of  the  Predator  (Cooter,  2001) 
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The  Predator  is  equipped  with  a  color  camera  in  the  nose  (generally  used  by  the 
pilot  for  flight  control),  a  day  variable-aperture  TV  camera,  a  variable-aperture  infrared 
(IR)  camera  (for  low  light/night),  and  a  synthetic  aperture  radar  (SAR)  for  looking 
through  smoke,  clouds  or  haze.  The  cameras  produce  full  motion  video  while  the  SAR 
produces  still-frame  radar  images.  Thus,  it  can  produce  near-real-time  multi-sensor 
imagery  and  intelligence.  The  latest  development  of  the  RQ/MQ-1  is  equipped  with 
lasers,  targeting  systems,  and  a  pair  of  hell  lire  missiles.  The  following  model  “MQ-9  is 
intended  to  fly  higher  and  faster,  provide  more  power,  and  carry  larger  payloads  than  the 
original  Predator  system.  It  is  intended  to  provide  a  more  robust  airframe,  using  a 
conventional  turbo-prop  engine  and  redundant  avionics.  MQ-9  will  be  used  primarily  in 
an  anned  reconnaissance  (“hunter-killer”)  role  and  will  perform  reconnaissance, 
surveillance,  and  target  acquisition  (RSTA)  as  a  secondary  role”  (Christie,  2003). 

One  has  to  understand  the  Predator  as  a  complex  system  which  includes 
•  4  x  Air  Vehicles 


Figure  16.  UAV  Predator  (Cooter,  2001) 
•  lx  Ground  Control  Station 


Figure  17.  Predator  Ground  Control  Station  -  outside  (Cooter,  2001) 
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Figure  18.  Predator  Ground  Control  Station  -  inside  (Cooter,  2001) 
•  lx  Predator  Primary  SATCOM  Link 


Figure  19.  Predator  Primary  SATCOM  Link  (Cooter,  2001) 

For  operating  the  Predator  a  HUD  is  being  used.  Figure  20  shows  the  current  one 
which  will  be  replaced  in  the  future  by  a  more  advanced  IHUD  (see  Figure  21). 
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Figure  20.  Heads-Up-Display  Predator  (Ball  et  al.,  2002) 


Figure  2 1 .  Heads-Up-Display  Predator  -  IHUD  (Guy,  2007c) 


The  Predator  HUD  can  be  considered  as  one  of  the  traditional  flight  instruments 


which  we  want  to  compare  with  the  new  prototype  display  design  in  the  present  research. 
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Next  to  the  HUD/  IHUD  the  pilot  uses  the  front  color  camera  for  operating  the 
Predator  (see  Figure  22  and  Figure  23). 


Figure  22.  View  from  the  color  camera  in  the  nose 
of  the  Predator  (Guy,  2007b) 


Figure  23.  View  from  the  color  camera  in  the  nose  of  the 
Predator  during  recovery  (Guy,  2007b) 
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D.  GROUND  CONTROL  STATION 

The  ground  control  station  (GCS)  is  the  control  center  of  any  UAV  system.  The 
main  tasks  that  have  to  be  accomplished  are  data  processing  of  the  incoming  data  and 
sending  control  signals  to  the  UAV.  Thus  three  main  features  will  be  executed  in  the 
GCS: 

•  Mission  planning:  The  purpose  of  mission  planning  is  to  analyze  available 
information  and  develop  a  flight  plan.  This  involves  identifying  and 
prioritizing  targets,  and  examining  sensor  capabilities  relative  to  the 
chosen  targets.  Weather  analysis  also  is  common. 

•  Mission  control:  Mission  control  is  responsible  when  the  aircraft  is  on  the 
ground,  when  it  is  getting  launched,  and  during  the  mission,  for  the 
navigation  and  possible  mid-flight  rerouting.  It  is  also  responsible  for 
monitoring  and  controlling  of  any  payload,  and  for  recovering  the  aircraft 
if  an  unusual  attitude  occurs. 

•  Data  manipulation:  Data  manipulation  is  responsible  for  processing, 
exploiting,  and  archiving  of  data  during  the  flight  and  /  or  after  the  flight 
(Anderson,  2002). 


The  Predator  GCS  provides  command  and  control  of  the  air  vehicle  through  the 
pilot  station  using  stick  and  rudder  control.  The  sensor  operators  monitor  the  flow  of 
information  from  the  UAV.  Their  sources  of  information  are  the  TV  camera,  the  infrared 
camera,  and  the  onboard  radar.  Depending  on  the  weather  conditions,  advantageously  the 
radar  can  be  operated  simultaneously  with  either  the  TV  camera  or  the  infrared  camera. 
For  communication  purposes,  a  C-band  line-of-sight  data  link  and  for  beyond-line-of- 
sight  a  Ku-band  satellite  data  link  are  used  (Anderson,  2002). 

The  following  pictures  give  an  idea  about  the  inside  of  a  GCS.  For  comparison 
reasons,  not  only  the  GCS  of  the  Predator  will  be  displayed,  but  also  several  others. 
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Figure  24.  GCS  Predator  -  Inside  view  (Cooter,  2006) 


Figure  25.  GCS  Predator  -  Inside  view 
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Figure  26.  GCS  Predator  -  Inside  view  (Cooter,  2006) 


Figure  27.  GCS  Pioneer  -  Inside  view 
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Figure  28.  GCS  Hunter  -  Inside  view 


Figure  29.  GCS  of  German  System  KZO  -  Inside  view 
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V.  METHOD 


A.  METHODOLOGY 

1.  Experiment  Goals  and  Purpose 

The  goals  of  this  study  are  to  evaluate  the  strengths  and  weaknesses  of  a  new 
proposed  display  design,  called  the  WEBER-Box,  compared  to  the  traditional  flight 
instruments. 


2.  Research  Questions 

The  study  is  limited  in  its  scope  and  assets.  We  wanted  to  answer  the  following 
questions: 

a)  Will  the  new  display  design  help  the  pilot  to  recognize  the  current  aircraft 
attitude? 

b)  Will  the  new  display  design  help  the  pilot  to  differentiate  between  a  usual 
and  unusual  attitude? 

c)  Will  the  new  design  help  to  prevent  misperception  of  an  unusual  attitude 
when  the  UAV  is  not  in  an  unusual  attitude? 


Answers  to  these  questions  reveal  whether  the  new  display  design  helps  to 

•  recognize  the  current  attitude  at  least  as  well  as  the  traditional  display 
design, 

•  differentiate  between  a  usual  and  an  unusual  attitude  at  least  as  well  as  the 
traditional  design,  and 

•  determine  whether  the  new  design  prevents  misjudging  an  unusual 
attitude. 

3.  Constraints  and  Assumptions 

Since  the  experiment  involved  human  participants,  the  plan  for  the  experiment 
was  submitted  to  the  Institutional  Review  Board,  which  approved  the  experiment  (see 
Appendix  C). 
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The  major  constraint  was  the  use  of  a  digital  hand  stopwatch,  type  SPORTLINE  ®  . 
Therefore  the  accuracy  of  measuring  the  time  to  judge  the  current  attitude  may  have  been 
decreased,  but  this  possible  disadvantage  was  constant  across  conditions  and,  presumably 
across  data  points  (1,120). 

The  major  assumption  was  about  the  definition  of  unusual  attitude.  According  to 
Air  Force  Instruction  11-217  “an  unusual  attitude  is  an  aircraft  attitude  occurring 
inadvertently”.  Since  this  is  very  broad,  we  followed  a  definition  according  to  industry 
...”as  one  where  pitch  unintentionally  exceeds  25  degrees  nose  up  or  10  degrees  nose 
down,  bank  angle  unintentionally  exceeds  45  degrees,  and  the  airspeed  is  inappropriate 
for  the  conditions”  (Guy,  2007a). 

4.  Technical  Equipment 

The  experiment  took  place  at  the  Naval  Postgraduate  School,  Monterey, 
California,  Watkins  Building,  room  212B. 

Two  workstations  were  used.  Workstation  one  was  for  instruction  and  training 
while  workstation  two  was  established  for  the  underlying  experiment. 

Workstation  one: 

The  computer-generated  images  were  displayed  on  a  19”  computer  monitor  with  a 
1024  x  768  pixel  resolution.  The  pictures  of  the  flight  situations  were  generated  by 
screenshots  of  the  commercial  flight  simulator  software  X-Plane™,  version  7.13  and 
version  8.40  and  by  screenshots  of  the  WEBER-Box  program,  respectively.  The  pictures 
were  stored  in  a  PowerPoint™  slideshow  (see  Appendix  D)  to  ensure  that  every 
participant  saw  the  different  attitudes  in  the  same  order. 

Workstation  two ; 

The  computer-generated  images  were  displayed  on  a  19”  computer  monitors  with 
a  1024  x  768  pixel  resolution.  The  second  screen  was  located  on  top  of  each  other  similar 
to  the  ground  station  of  a  Predator.  It  provided  during  the  duration  of  the  experiment  the 
definition  of  unusual  attitude. 
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The  simulation  of  the  flight  situations  were  generated  by  the  commercial  flight 
simulator  software  X-Plane™,  version  7.13  and  version  8.40  and  by  the  WEBER-Box 
program,  respectively.  The  predefined  simulations  of  usual  and  unusual  attitude  will  be 
stored  in  Camtasia  Studio™  from  TechSmith  to  ensure  that  every  participant  saw  exactly 
the  same  the  different  attitudes  in  his  randomly  assigned  order. 

5.  Data  Collection  Methodology 

Several  types  of  data  were  collected  during  the  experiment. 

First,  demographic  data  were  collected  by  a  pre-experiment  questionnaire. 
Secondly  data  were  collected  during  the  experiment.  These  were  data  for  correct/ 
incorrect  responses,  and  the  measured  time  each  participant  needed  to  judge  the  current 
attitude.  In  addition,  the  use  of  subjective  self-assessment  provided  a  scale  of  five 
judgments  in  every  questionnaire. 

In  summary,  a  “repeated  measures”  design  was  used  in  which  all  the  participants 
were  exposed  to  all  the  instrument  setups  and  all  the  sub-experiments.  The  order  of 
presentation  of  the  two  displays  and  the  color-coding  was  pseudo-randomized  (see 
Appendix  J  and  K).  This  procedure  ensured  that  learning  effects  would  be 
counterbalanced  over  the  experiment. 

6.  Data  Analysis  Methodology 

The  data  was  hand  recorded  by  using  a  template  (see  Appendix  K).  The 
participant  judged  the  current  attitude  and  responded  by  saying  “Usual  attitude”  or 
“Unusual  attitude”.  The  experimenter  measured  the  time  and  transferred  the  time  into  the 
template  together  with  the  participants’  verbal  response  of  his  judgment. 

The  data  then  were  analyzed  by  commercial  software  Excel  2003  and  SPSS  13.0 
for  Windows.  Methods  that  have  been  used  were  Analysis  of  Variance  and  the  paired  t- 
test.  The  statistical  analysis  concentrated  on  the  differences  between  the  measurements 
depending  on  the  experimental  condition:  display  type  (traditional  or  WEBER-Box)  and 
color  coding  (none  or  yellow). 
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Two  questionnaires  were  handed  out.  The  pre-experiment  questionnaire  provided 
demographic  information  and,  for  pilots,  aviation  background.  The  post-experiment 
questionnaire  was  helpful  as  a  secondary  method  for  determining  the  effects  of  the 
different  variables  display  design  and  color/  no  color,  and  to  analyze  the  relationship 
between  subjective  and  quantitative  data. 


B.  EXPERIMENTAL  DESIGN 

1.  Basic  Experimental  Design 

The  experiment  was  designed  to  compare  the  correct  responses  of  the  judgment  of 
usual  or  unusual  attitude  while  changing  the  variables  display  design  and  color  coding  or 
not.  Therefore,  it  was  a  randomized  “within  subjects”  design  because  every  participant 
received  the  same  treatment.  Since  two  groups,  pilots  and  non-pilots,  were  involved  in 
the  experiment,  their  comparison  was  “between  subjects.” 

The  dependent  variables  were  the  outcome  of  correct  response  and  the  time  a 
participant  needed  to  judge  the  attitude.  Independent  variables  were  the  display  design 
and  the  factor  color  or  no  color. 

Each  participant  was  told  at  the  beginning  that  his  duty  station  is  the  ground 
control  station  of  a  Predator. 

2.  Experimental  Procedure 

The  overall  experiment  was  subdivided  into  four  phases. 

Phase  one  —  Pre -Paperwork: 

The  participant  filled  out  the  necessary  paperwork,  including  a  consent  form,  a 
privacy  act  statement,  a  minimal-risk  information  form,  and  an  initial/  demographic 
questionnaire. 
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Phase  two  —  Introduction  and  Training 

Then  the  participant  was  introduced  to  the  experiment.  In  a  PowerPoint 
presentation  he  received  infonnation  about  the  UAV,  his  task  as  an  operator  of  the  UAV, 
and  details  about  usual  and  unusual  attitude  since  this  was  the  key  part  of  the  underlying 
experiment.  The  participant  also  watched  a  2-minute  video  of  the  work  as  an  operator  in 
the  ground  control  station  of  the  Predator. 

This  pre-experimental  part,  phases  one  and  two,  took  approximately  25  minutes. 

Phase  three  —  Experiment 

After  completion  of  introduction  and  training,  the  participant  started  with  the  first 
experimental  sub-task,  given  a  particular  simulation  setup,  that  is,  with  or  without  the 
new  instrument.  Each  of  the  14  trials  of  the  sub-experiment  was  scheduled  for  10 
seconds.  Two  monitors  were  placed  on  top  of  each  other  to  create  an  environment  like  a 
ground  station  of  a  UAV.  The  top  monitor  was  only  used  for  displaying  the  definition  of 
unsual  attitude.  The  first  two  trials  in  each  combination  always  were  practice  trials. 

The  experimenter  started  the  prepared  simulation  of  19  different  flight  simulations 
that  ended  in  a  usual  or  unusual  attitude.  Each  simulation  appeared  for  ten  seconds.  This 
was  also  the  amount  of  time  given  to  the  participant  to  judge  the  attitude.  After  ten 
seconds,  the  next  attitude  appeared  automatically.  One  half  of  the  trails  were  created  as 
usual  attitude  while  the  other  half  as  unusual  attitude. 

After  accomplishing  the  first  two  blocks  of  trials,  the  participant  was  allowed  a  5- 
minute  break  if  desired. 

Then  the  participant  started  the  second  sub-experimental  task. 
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Intro  and 
Initial 
Briefings 


Break 


Debriefing 


Figure  30.  Experimental  setup 


In  summary,  all  the  participants  were  exposed  to  all  the  instrument  setups  and  all 
the  sub-experiments.  Each  participant  was  confronted  with  seven  trails  of  usual  attitude 
with  traditional  design  and  seven  trails  of  unusual  attitude  with  the  traditional  design  and, 
likewise  with  the  new  design  (see  Figure  30). 

Phase  four  —  Post-Paperwork 

When  finished,  the  participant  was  asked  to  fill  out  the  post-experimental 
questionnaire  and  support  the  experiment  with  his  suggestions  and  connnents.  This 
completed  the  experimental  session. 


C.  PILOT  STUDY  ONE 

The  first  pilot  study  was  conducted  with  three  participants  to  test  the  procedures, 
apparatus,  and  tasks.  The  setup  of  this  pilot  study  differed  from  the  later  conducted 
experiment.  It  was  a  dynamic  environment  with  a  flight  simulation  that  stopped  after  120 
seconds  and  the  participant  had  to  judge  the  current  attitude.  For  this  judgment  he  was 
given  ten  seconds.  There  was  also  only  the  comparison  between  the  traditional  display 
design  without  color  coding  and  the  new  display  design  with  color  coding.  Two  of  the 
three  participants  claimed  that  this  was  a  disadvantageous  setup  due  to  the  outstanding 
support  the  new  display  design  received.  The  recommendations  were  to  extend  the  setup 
and  provide  both  of  the  display  design  types  with  color  and  compare  the  results. 
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D.  PILOT  STUDY  TWO 


Therefore  another  pilot  study  had  to  be  conducted  according  to  the  above 
mentioned  recommendations.  Again  three  participants  tested  the  procedures,  apparatus, 
and  tasks.  The  new  setup  contained  now  both  display  designs,  each  with  and  without 
color  coding.  We  also  changed  the  settings  contrary  to  the  former  pilot  study  to  a  static 
environment  with  a  presentation  mode  where  the  participant  had  to  judge  the  attitude  in  a 
maximum  time  of  ten  seconds.  If  he  was  unable  to  judge  in  this  period,  it  was  counted  as 
incorrect.  All  of  the  participants  were  tested  on  each  of  the  four  combinations.  The  pilot 
study  demonstrated  that  every  participant  was  able  to  judge  the  current  attitude  and  the 
given  time  of  ten  seconds  sufficed. 


E.  PARTICIPANTS 

The  participants  were  randomly  assigned  on  a  volunteer  basis  from  students  and 
faculty  of  NPS.  The  study  involved  20  participants,  evenly  partitioned  into  ten  pilots  and 
ten  non-pilots.  Eight  of  the  pilots  were  jet  pilots  and  two  were  helicopter  pilots  with  an 
average  of  1,348  flight  hours.  Therefore  the  group  of  pilots  can  be  considered  as 
participants  with  a  strong  aviation  background  and  a  high  flight  experience.  The  mean 
age  of  the  pilots  was  36  in  the  range  from  28  to  42.  The  mean  age  of  the  non-pilots  was 
34  with  a  range  from  28  to  42. 
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VI.  DATA  ANALYSIS 


A.  HYPOTHESIS 

Inferential  testing  techniques  enable  the  researcher  to  decide  between  the  null 
hypothesis  and  the  alternative  hypothesis.  Devore,  2004,  defines  the  null  hypothesis, 
“denoted  by  a ,  as  the  claim  that  is  initially  assumed  to  be  true.”  The  alternative  or 
secondary  hypothesis,  “denoted  by  Ha ,  is  the  assertion  that  is  contradictory  to  H0 .  The 
null  hypothesis  will  be  rejected  in  favour  of  the  alternative  hypothesis  only  if  sample 
evidence  suggests  that  H0  is  false.  If  the  sample  does  not  strongly  contradict  H0,  we  will 
continue  to  accept  the  null  hypothesis.  The  two  possible  conclusions  from  a  hypothesis¬ 
testing  analysis  are  then  reject  H0  or  fail  to  reject  H0”  (Devore,  2004). 

A  test  procedure  is  specified  by  the  following: 

•  A  test  statistic,  a  function  of  the  sample  data  on  which  the  decision  (reject 
H0  or  do  not  reject  H0)  is  to  be  based. 

•  A  rejection  region,  the  set  of  all  statistic  values  for  which  H0  will  be 
rejected. 

The  null  hypothesis  H0  will  then  be  rejected  if  and  only  if  the  observed  or 
computed  test  statistic  value  falls  in  the  rejection  region  (Devore,  2004).  Evidence  to 
reject  a  null  hypothesis  H0  is  given  if  the  p-value  is  small  enough  (the  probability  that  a 
difference  of  the  magnitude  observed  may  have  occurred  by  chance). 

Since  we  were  testing  two  different  measures  of  effectiveness  (MOE),  two 
different  null  hypotheses  were  stated  along  with  their  corresponding  alternative 
hypothesis.  Our  dependent  variables,  or  MOEs,  were  the  “probability  of  correct 
response”  and  its  corresponding  “time  of  response”. 

1.  MOE  “Probability  of  Correct  Response” 
a.  Null  Hypothesis 

H0 :  no  difference  between  traditional  and  prototype  display  design 
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b.  Alternative  Hypothesis 

Ha :  prototype  display  design  provides  a  higher  percentage  of  correct 


response 

Null  or  alternative  hypotheses  can  be  explored  by  using  the  results  of  the  binary 
data  ( 1  for  correct  response  and  0  for  incorrect  response)  of  the  experiment  when  the  test 
participant  had  to  judge  the  current  attitude. 

2.  MOE  “Time  of  Response” 

a.  Null  Hypothesis 

H0:  no  difference  in  response  time  between  traditional  and  prototype 
display  design 

b.  Alternative  Hypothesis 

Ha :  prototype  display  enables  to  respond  correctly  in  a  shorter  time 

The  second  null  or  alternative  hypotheses  can  be  explored  by  using  the  results  of 
the  time  measurement  in  seconds  and  milliseconds  of  each  judgement  task.  The  time  each 
participant  required  to  judge  his  attitude,  indicates  how  fast  he  can  build  his  model  of  the 
current  spatial  situation  and  therefore  being  able  to  start  a  recovery  if  necessary.  Each 
participant  is  tested  on  the  traditional  and  the  prototype  display  design.  The  order  of 
presentation  was  counterbalanced.  The  data  provides  an  indication  about  which  display 
design  enables  a  person  to  assess  an  aircraft  attitude  more  rapidly. 

In  addition,  the  post-experiment  questionnaire  provides  us  with  feedback  for  both 
measures  of  effectiveness.  Assuming  a  faster  response  to  the  judgement  task  when  using 
the  prototype  display  design,  this  should  be  reflected  by  selecting  an  answer  of  “border 
line”,  “somewhat  more”  or  “much  more”  with  regards  to  better  awareness  of  spatial 
orientation. 

Additionally,  an  answer  to  the  question  of  the  influence  of  a  color  coded  support 
for  pitch  and/  or  roll,  either  for  the  traditional  or  the  prototype  display  design,  can  be 
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extracted  from  the  questionnaire.  By  choosing  any  answer  above  “border  line”,  we  can 
assume  that  the  participant  favoured  an  additional  color  support. 


3.  Baseline  Determination 

Before  starting  with  any  experiment,  it  is  necessary  to  evaluate  the  number  of 
participants  necessary  to  achieve  a  statistically  significant  outcome. 

Kanji  (1999)  provides  a  general  overview  of  the  relation  between  a  true  or  false 
hypothesis  and  when  to  reject  or  not  to  reject  this  hypothesis.  The  two  ways  of  making 
mistakes  when  perfonning  a  hypothesis  test  are  type  I  and  type  II  errors.  Type  I  errors 
represent  mistakenly  rejecting  H0,  when  H0  is  true.  On  the  other  hand,  an  error  of  type 

II  occurs  when  H0  is  false  and  it  is  not  rejected. 

The  goal  is  to  avoid  type  I  and  type  II  errors  and  make  the  correct  decision  about 
accepting  or  rejecting  the  null  hypothesis. 


What  is  true  in  the  population? 


Conclusion  No  effect 

reached  in  a 

study 

Treatment  effect 


Treatments  have  Treatments  have 

no  effect  an  effect 


Correct  decision 

(p  =  l-/?) 

Type  II  error 

(p  =  Ci) 

Type  I  error 

(P=  a) 

Correct  decision 

(P  =  l  — /?) 

Power 

Table  2.  Outcomes  of  statistical  tests  (Murphy  &  Myors,  2004) 

The  power  of  a  statistical  test  is  defined  as  the  probability  of  rejecting  the  null 
hypothesis,  H0t  when  the  alternative  hypothesis  is  true  (Montgomery  &  Runger,  2007). 
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Instead  of  simply  calculating  the  level  of  power,  it  is  also  possible  to  determine 
the  sample  size  to  achieve  specific  levels  of  power.  By  convention,  it  is  necessary  to 
achieve  at  least  a  power  of  0.05,  otherwise  the  probability  is  greater  than  .05  that  a  Type  I 
error  will  occur,  i.e.,  failing  to  reject  H0  when  H0  is  false.  Achieving  a  power  greater 
than  0.08  is  always  desirable  (Murphy  &  Myors,  2004). 

We  used  a  power  analysis  and  determination  of  the  sample  size  according  to 
DeVeaux  et  ah,  2005.  By  regrouping  the  formula 


from: 


to: 


ME  =  t 


n- 1 


■sjn  = 


ME 


n  = 


/  *  \2 

'  t„-l  '  5  ^ 

V  ME  J 


it  is  possible  to  detennine  the  sample  size. 


Explanation:  ME:  margin  of  error 

extent  of  an  interval  on  either  side 
fn_x  ■  confidence  interval 

an  interval  estimate  for  a  population  parameter 
s:  standard  error 

estimate  of  the  standard  deviation  of  a  sampling 
distribution 

calculated  by  the  product  of  p  x  q  with  p  as  the  probability 
that  the  observed  statistic  value  could  occur  and  q  as  the 
probability  that  the  observed  statistic  value  could  not  occur 


To  determine  an  appropriate  sample  size,  we  made  the  following  assumptions. 
The  level  of  significance  will  be  a  =  0.05  and  the  probability  of  90%  that  the  statistic 
value  will  occur.  The  results  shown  in  Table  2  were  calculated  by  using  the  above 
displayed  formula  in  a  spreadsheet  in  Microsoft  Excel  2003. 
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ME 

P 

Q 

z 

Sample  size  n 

Cl 

0.03 

0.9 

0.1 

1.282 

14.79 

80% 

0.04 

0.9 

0.1 

1.645 

13.70 

90% 

0.04 

0.9 

0.1 

1.96 

19.45 

95% 

0.05 

0.9 

0.1 

2.326 

17.53 

98% 

0.05 

0.9 

0.1 

2.576 

21.50 

99% 

Table  3.  Number  of  proposed  participants 


Therefore  we  designed  the  experiment  for  20  participants,  assuming  a  margin  of 
error  of  0.04  with  a  p-value  of  90%  using  a  confidence  interval  of  95%.  Since  we  decided 
to  choose  a  level  of  significance  of  a  =  0.05,  H0  can  be  rejected  if  the  p-value  is  less 
than  0.05. 


B.  GENERAL  DATA  ANALYSIS 

1.  General  Data  Description 

During  the  experiment  1,120  data  points  (20  participants  x  4  trial  x  14  tests  = 
1,120)  in  different  categories  were  collected.  The  following  software  was  used  for  the 
data  analysis:  Microsoft  Excel  2003  and  SPSS  13.0  for  Windows.  These  programs  have 
also  been  used  for  the  graphical  output  of  the  analysis. 

The  data  categories  were: 

1)  Initial  Questionnaire:  The  questionnaire  was  also  used  to  obtain 
demographic  data  like  age  in  years  and  flight  experience  in  hours. 

2)  Judgement  of  the  current  attitude:  The  answer  to  the  task  of  judging  the 
current  attitude  as  correct  or  incorrect. 

3)  Judgement  Time:  The  time  the  participant  needed  to  judge  the  current 
attitude  in  sec. 

2)  Post  Experiment  Questionnaire:  The  subjective  assessment  of  the 
experiment. 
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2.  Demographic  Data 

a.  Age 

Both  pilots  and  non-pilots  had  a  similar  mean  age.  The  standard  deviation 
of  the  non-pilots  was  a  little  higher  due  to  more  participants  of  40  years  and  older.  The 
detailed  statistics  are  displayed  in  Table  4. 


Mean 

STD 

Minimum 

Maximum 

Pilots 

35.50 

3.85 

28 

42 

Non-pilots 

34.00 

4.77 

28 

42 

Total 

34.75 

4.40 

28 

42 

Table  4.  Statistics  on  participants’  age 


b.  Flight  Experience 


ID- 

Participant 

Pilot 

(1/0) 

Flight- 

Hours 

Mean 

Flight- 

Hours 

STD 

001 

1 

80.00 

002 

1 

20.00 

003 

1 

2000.00 

004 

1 

2500.00 

005 

1 

1400.00 

006 

1 

2400.00 

007 

1 

10.00 

008 

1 

1971.00 

101 

1 

2000.00 

102 

1 

1100.00 

1348.10 

943.29 

Table  5.  Flight-Hours  -  Raw  data 


Flight  Hours 


ID  -  Participant 


Figure  3 1 .  Chart  of  individual  flight-hours 
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Seven  pilots  out  of  the  total  of  ten  pilots  had  more  than  1,000  flight  hours 
and  four  out  of  these  seven  pilots  had  more  than  2,000  flight  hours,  with  one  participant 
very  close  to  the  2,000  flight  hour  barrier,  one  can  consider  the  group  of  pilots  as 
experienced.  Most  of  them  had  some  experience  with  SD  during  their  flying  career.  Table 
6  provides  the  descriptive  statistics  of  the  pilot’s  flight  experience. 


Mean 

STD 

Minimum 

Maximum 

Flight  experience 

1,348.10 

943.29 

10 

2,500 

Table  6.  Statistics  on  pilot’s  flight  experience 


3.  General  Statistical  Methodology 

We  tested  each  participant  on  the  factors  “display  design”  and  “color  support/  no 
color  support.”  In  addition,  there  were  two  types  of  participants,  pilots  and  non-pilots, 
yielding  a  full  factorial  design  with  2J  =  8  factor-level  combinations.  Our  measures  of 
effectiveness  were  correct  answer  of  aircraft  attitude  (usual  or  unusual  attitude)  and  the 
time  a  participant  needed  to  make  that  judgment  (see  Figure  32  and  Figure  33). 

Analysis  of  variance  (ANOVA)  was  used  to  test  the  effects  of  the  independent 
variables  on  the  correct  judgment  of  attitude  and  the  time  to  respond. 
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Figure  32.  Combination  and  evaluation  of  correct  response 
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Figure  33.  Combination  and  evaluation  of  time  to  judge 
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4.  Baseline  Comparison 

The  basic  data  are  displayed  in  the  middle  section  of  Figure  32  and  Figure  33  and 
combined  in  Table  7  with  the  raw  data. 


ID- 

Participant 

Pilot 

(1/0) 

prob.  correct 

mean  time 

(Traditional  + 
Non-Color) 

prob.  correct 

mean  time 

(Traditional  + 
Color) 

prob.  correct 

mean  time 

(Prototype  + 
Non-Color) 

prob.  correct 

mean  time 

(Prototype  + 
Color) 

response 

(Traditional  + 
Non-Color) 

response 
(Traditional  + 
Color) 

response 

(Prototype  + 
Non-Color) 

response 

(Prototype  + 
Color) 

001 

1 

1.00 

1.44 

1.00 

0.66 

0.93 

0.91 

1.00 

0.68 

002 

1 

0.57 

3.07 

1.00 

1.28 

1.00 

2.22 

1.00 

1.31 

003 

1 

0.86 

1.74 

0.79 

2.32 

1.00 

1.54 

1.00 

1.45 

004 

1 

1.00 

1.68 

1.00 

1.19 

1.00 

1.06 

1.00 

1.80 

005 

1 

0.86 

2.02 

1.00 

0.77 

1.00 

1.30 

1.00 

0.72 

006 

1 

0.93 

1.91 

1.00 

1.77 

1.00 

1.05 

1.00 

1.15 

007 

1 

0.86 

1.44 

1.00 

1.63 

1.00 

1.07 

1.00 

1.03 

008 

1 

1.00 

1.70 

1.00 

1.74 

1.00 

1.93 

1.00 

1.20 

101 

1 

1.00 

1.62 

1.00 

2.28 

1.00 

1.40 

1.00 

0.89 

102 

1 

1.00 

2.29 

1.00 

2.02 

1.00 

2.00 

1.00 

1.36 

201 

0 

1.00 

2.31 

1.00 

1.74 

1.00 

1.83 

1.00 

1.04 

202 

0 

0.93 

1.46 

0.86 

1.88 

0.93 

1.25 

1.00 

0.83 

203 

0 

0.86 

2.02 

1.00 

0.88 

1.00 

1.57 

1.00 

0.96 

204 

0 

0.86 

2.22 

0.93 

1.22 

0.86 

1.79 

1.00 

0.84 

205 

0 

1.00 

3.06 

1.00 

3.27 

1.00 

2.20 

1.00 

1.84 

206 

0 

0.71 

4.89 

1.00 

0.98 

1.00 

1.74 

1.00 

1.19 

207 

0 

0.93 

2.40 

1.00 

1.10 

1.00 

1.83 

1.00 

0.79 

208 

0 

0.86 

3.02 

1.00 

0.96 

1.00 

1.38 

1.00 

0.89 

209 

0 

1.00 

2.83 

1.00 

1.03 

1.00 

2.21 

1.00 

0.90 

210 

0 

0.86 

3.85 

1.00 

1.03 

0.93 

2.19 

1.00 

0.91 

1  MEAN:  || 

0.90 

JL 

2.35 

Jl 

0.98 

JL 

1.49 

0.98 

1.62 

1.00 

1.09  | 

1  STD:  1! 

0.11 

□L 

0.87 

□  1 

0.05 

□L 

0.63 

□L 

0.04 

□L 

0.42 

=c 

0.00 

=c 

0.32  1 

Table  7.  Baseline  comparison  -  Raw  data 


C.  ANALYSIS  OF  VARIANCE 
1.  General  Linear  Model 

Analysis  of  variance  (ANOVA)  is  an  analysis  method  for  testing  equality  of 
means  across  three  or  more  treatment  groups.  In  our  case  we  used  a  one-way  ANOVA  for 
repeated  measures.  On  one  hand  we  have  a  “within  subjects”  (see  Table  8  and  Table  9) 
design  because  every  participant  received  the  same  treatment. 


53 


Within-Subjects  Factors 


Measure:  MEASURE  1 


display  color 

Dependent 

Variable 

1  1 

2 

prob. 

correctrespo 

nse 

Traditional 

NonColor 

prob. 

correctrespo 

nse 

Traditional 

Color 

2  1 

2 

prob. 

correctrespo 

nse 

Prototype 

NonColor 

prob. 

correctrespo 

nse 

Prototype 

Color 

Table  8.  Within  subjects  for  probability  of  correct  response 


Within-Subjects  Factors 


Measure:  MEASURE  1 


display  color 

Dependent 

Variable 

1  1 

2 

meantime 

Traditional 

NonColor 

meantime 

Traditional 

Color 

2  1 

2 

meantime 

Prototype 

NonColor 

meantime 

Prototype 

Color 

Table  9.  Within  subjects  for  mean  time 
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The  two  groups  of  participants,  pilots  and  non-pilots,  were  a  “between  subjects” 


factor  (see  Table  10). 


Between-Subjects  Factors 


N 

Pilot 

0 

10 

(1/0) 

1 

10 

Table  10.  Between  subjects 


Therefore  we  are  able  to  assess  the  effect  of  display  design  or  color  versus  the 
status  as  pilot  or  non-pilot. 


2.  ANOVA  for  Probability  of  Correct  Response 

The  ANOVA  for  probability  of  correct  response  gave  us  the  statistical  output 
shown  in  Table  11. 


Tests  of  Within-Subjects  Contrasts 


Measure:  MEASURE  1 


Source 

display 

color 

Type  III  Sum 
of  Squares 

df 

Mean  Square 

F 

Siq. 

display 

EBB 

.049 

1 

.049 

11.130 

.004 

display  *  PilotIO 

D 

.000 

1 

.000 

.056 

.816 

Error(display) 

EBB 

.079 

18 

.004 

color 

Linear 

.042 

1 

.042 

10.805 

.004 

color  *  PilotIO 

.001 

1 

.001 

.250 

.623 

Error(color) 

.071 

18 

.004 

display  *  color 

Linear 

.016 

1 

.016 

3.540 

.076 

display  *  color  *  PilotIO 

Linear 

.000 

1 

.000 

.053 

.820 

Error(display*color) 

Linear 

.083 

18 

.005 

Table  11.  Test  of  within-subjects  contrasts  for  probability  of  correct  response 


The  effect  of  display  design  (traditional  or  Weber  Box)  was  highly  significant,  F 
=  11.13,  p  =  0.004.  Likewise,  the  effect  of  color  coding  was  significant,  F  =  10.81,  p  = 
.004.  There  were  no  significant  two-way  interactions  between  display  design  and  color  (p 
=  0.076),  display  design  and  pilot/  no  pilot  (p  =  0.816),  or  color  and  pilot/  no  pilot  (p  = 
0.623).  Similarly,  the  three  way  interaction  between  display,  color,  and  pilot  was  not 
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significant  (p  =  0.820).  Figure  34  shows  the  nearly-significant  interaction  between 
display  type  and  color.  It  can  also  be  seen  in  the  figure  that  response  accuracy  was  higher 
for  the  WEBER  Box  than  for  the  traditional  display,  both  with  and  without  color. 


Estimated  Marginal  Means  of  MEASURE_1 


color 

- no  color 

- color 


Figure  34.  Interactions  of  display  design  and  color 


Participants  using  the  WEBER-Box  reached  a  probability  of  correct  response  of 
98%  without  color  coding  and  100%  with  color  coding.  These  results  indicate  that  high 
accuracy  in  judging  attitude  is  achieved  using  the  WEBER  Box. 


a.  T-Test:  Traditional  Design  Versus  Color 

In  support  of  the  ANOVA  findings,  a  Most  was  perfonned  to  determine 
whether  color  coding  had  an  effect  on  response  accuracy  with  the  traditional  display  (see 
Table  12).  The  significant  result,  p  <  .05,  indicates  that  response  accuracy  using  the 
traditional  display  was  better  with  color  coding  than  without  it.  The  boxplot  also  supports 
this  conclusion  (see  Figure  35). 
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prob.  correct  response 
(Traditional  without  color) 

prob.  correct  response 
(Traditional  with  color) 

Mean 

0.9045 

0.979 

Variance 

0.012552368 

0.003146316 

Observations 

20 

20 

P(T<=t)  one-tail 

0.005702026 

P(T<=t)  two-tail 

0.011404052 

Table  12.  Analysis  in  the  probability  of  correct  response  between  traditional 

design  without  color  vs.  with  color 


prob.  correct  response  (Traditional  +  Non-Color)  prob.  correct  response  (Traditional  +  Color) 


Figure  35.  Graphical  output  of  the  analysis  in  the  probability  of 
correct  response  between  traditional  design 
without  color  vs.  with  color 

3.  ANOVA  for  Response  Time 

The  ANOVA  for  response  time  resulted  in  the  statistical  output  shown  in  Table 
13. 
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Tests  of  Within-Subjects  Contrasts 


Measure:  MEASURE  1 


Source 

display 

color 

Type  III  Sum 
of  Squares 

df 

Mean  Square 

F 

Siq. 

display 

Kfima 

6.311 

1 

.000 

display  *  PilotIO 

d mm 

.374 

1 

.374 

3.479 

.079 

Error(display) 

BB 

1.935 

18 

.108 

color 

Linear 

9.737 

1 

9.737 

23.978 

.000 

color  *  PilotIO 

Linear 

3.054 

1 

3.054 

7.520 

.013 

Error(color) 

7.310 

18 

.406 

display  *  color 

Linear 

.533 

1 

.533 

1.951 

.179 

display  *  color  *  PilotIO 

Linear 

.422 

1 

.422 

1.544 

.230 

Error(display*color) 

Linear 

4.918 

18 

.273 

Table  13.  Test  of  within-subjects  contrasts  for  mean  response  time 


Display  type  (traditional  versus  WEBER  Box)  had  a  highly  significant  effect  on 
mean  response  time,  F  =  58.70,  p  <  001.  Similarly,  color  coding  had  a  highly  significant 
effect  on  response  time,  F  =  23.98,  p  <  .001. 

The  two  way  interaction  of  display  type  and  pilot/  non-pilot  did  not  reach 
significance,  F  =  3.48,  p  =  .079,  nor  did  the  interaction  of  display  type  and  color  coding, 
F  =  1.95,  p  =  .179.  On  the  other  hand,  the  interaction  of  color  coding  and  pilot/  non-pilot 
was  significant,  F  =  7.52,  p  <  .05.  The  three  way  interaction  of  display  type,  color  coding, 
and  pilot  was  not  significant,  F  =  1 .54,  p  =  .230. 

A  graphical  representation  of  display  type  and  color  coding  is  given  in  Figure  36. 
It  can  be  seen  in  the  figure  that  response  times  were  shorter  with  the  new  display  design 
(WEBER  Box)  than  with  the  traditional  instrument  design,  and  that  held  true  for  both 
color  coded  and  no  color  coded  displays. 
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Estimated  Marginal  Means  of  MEASURE_1 


color 

- no  color 

- color 


Figure  36.  Non-significant  interaction  between  display  design  and  color 


The  mean  time  decreased  when  switching  from  the  traditional  display  to  the 
WEBER-Box.  Adding  color  to  the  displays  made  this  decrease  even  more  obvious. 

Figure  37  shows  the  significant  interaction  between  display  design  and  pilot/  non¬ 
pilot.  Both  pilots  and  non-pilots  responded  more  quickly  with  the  WEBER  Box  than  with 
the  traditional  instrument  design,  but  the  advantage  of  the  WEBER  Box  was  greater  for 
the  non-pilots.  This  result  is  not  surprising  because  pilots  have  considerable  experience 
with  the  traditional  display  design,  whereas  the  non-pilots  were  quite  slow  in  responding 
to  the  traditional  display. 
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Estimated  Marginal  Means  of  MEASURE_1 


Pilot  (1/0) 

- no  pilot 

- pilot 


Figure  37.  Interactions  between  color  and  pilots 


This  graphical  outcome  shows  at  once  that  the  non-pilots  profited  most  from  the 
WEBER-Box.  Both,  pilots  and  non-pilots,  could  decrease  their  mean  response  time  when 
tested  on  the  WEBER-Box.  But  the  non-pilots  mean  response  time  dropped  in  such  a 
significant  way  even  underneath  the  pilots  value  when  using  the  WEBER-Box. 

This  is  an  additional  proof  of  the  superiority  of  the  WEBER-Box.  On  the  other 
hand  there  is  room  fur  further  investigations,  why  the  pilots  didn’t  benefit  as  much  from 
the  WEBER-Box  as  the  non-pilots. 


a.  T-Test:  Pilot  Versus  Display  Design 

In  support  of  the  ANOVA  findings  a  /-test  was  perfonned  to  detennine 
whether  display  design  had  an  effect  for  mean  response  time  on  pilots  (see  Table  14).  The 
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significant  result,  p  <  .05,  indicates  that  the  mean  response  time  of  the  pilots  using  the 
WEBER-Box  was  better  than  with  the  traditional  design.  The  boxplot  also  supports  this 
conclusion  (see  Figure  38). 


mean  time 

mean  time 

(Pilot  with  trad,  design) 

(Pilot  with  WEBER-Box) 

Mean 

1.729 

1.329 

Variance 

0.136054444 

0.123676667 

Observations 

10 

10 

P(T<=t)  one-tail 

0.011579295 

P(T<=t)  two-tail 

0.023158589 

Table  14.  Analysis  of  the  response  time  between  pilots  with  the 
traditional  design  vs.  the  WEBER-Box 


Figure  38.  Graphical  output  of  the  analysis  of  the  mean  response  time 

for  pilots  with  the  traditional  design 
vs.  the  WEBER-Box 


In  support  of  the  ANOVA  findings  another  /-test  was  performed  to  determine 

whether  display  design  had  an  effect  for  mean  response  time  for  non-pilots  (see  Table 
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15).  The  significant  result,  p  <  .05,  indicates  that  the  mean  response  time  of  the  non-pilots 
using  the  WEBER-Box  was  much  better  than  with  the  traditional  design.  The  boxplot 
also  supports  this  finding  (see  Figure  39). 


mean  time 

mean  time 

(Non-Pilot  with  trad. 

(Non-Pilot  with  WEBER- 

design) 

Box) 

Mean 

2.109 

1.407 

Variance 

0.32061 

0.074445556 

Observations 

10 

10 

P(T<=t)  one-tail 

0.001190929 

P(T<=t)  two-tail 

0.002381859 

Table  15.  Analysis  of  the  response  time  for  non-pilots  with  the 
traditional  design  vs.  the  WEBER-Box 


Figure  39.  Graphical  output  of  the  mean  response  time  (sec) 
for  non-pilots  with  the  traditional  design 
vs.  the  WEBER-Box 
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b.  T-Test:  Pilot  Versus  Color 

In  support  of  the  ANOVA  findings  a  /-test  was  perfonned  to  determine 
whether  color  had  an  effect  on  response  time  for  pilots  (see  Table  16).  The  significant 
result,  p  <  .05,  indicates  that  the  mean  response  time  of  the  pilots  using  the  WEBER-Box 
was  faster  than  with  the  traditional  design.  The  boxplot  also  supports  this  finding  (see 
Figure  40). 


mean  time 

mean  time 

(Non-Pilot  without  color) 

(Pilot  with  color) 

Mean 

2.305 

1.214 

Variance 

0.336538889 

0.249982222 

Observations 

10 

10 

P(T<=t)  one-tail 

0.000137018 

P(T<=t)  two-tail 

0.000274036 

Table  16.  Analysis  of  the  response  time  for  non-pilots  with 
color  vs.  without  color 


Figure  40.  Graphical  output  of  the  response  time  (sec)  of  non-pilots 

with  color  vs.  without  color 
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D.  POST-EXPERIMENT  QUESTIONNAIRE 

After  the  primary  data  collection  was  finished,  each  participant  was  asked  to  fill 
out  the  post-experiment  questionnaire.  This  questionnaire  served  as  an  indicator  of  the 
attitude  of  the  participants  toward  the  two  different  display  designs  and  the  difference  of 
a  colored-coded  support.  It  also  provided  subjective  data  in  relation  to  the  correct/ 
incorrect  answers  and  the  time  each  participant  needed  to  judge  the  various  aircraft 
attitudes.  The  raw  data  from  the  questionnaire  can  be  seen  in  Table  17. 


Difficulty  of 
Task 

Difficulty 
judging  by 
Traditional 

Difficulty 
judging  by 
Prototype 

Benefits  of 
Color-coding 

Better  aware 

with 

Prototype 
without  Color 

Better  aware 

with 

Prototype 
with  Color- 
Coding 

Prototype  in- 
/  decrease 
monitoring 
demands 

Coding 

1-5 

1-5 

1-5 

1-5 

1-5 

1-5 

1-5 

001 

3.00 

3.00 

2.00 

1.00 

4.00 

3.00 

2.00 

002 

3.00 

4.00 

2.00 

1.00 

4.00 

4.00 

2.00 

003 

2.00 

2.00 

2.00 

1.00 

3.00 

4.00 

2.00 

004 

1.00 

2.00 

1.00 

2.00 

4.00 

4.00 

2.00 

005 

4.00 

4.00 

2.00 

2.00 

5.00 

4.00 

1.00 

006 

1.00 

1.00 

1.00 

1.00 

3.00 

3.00 

3.00 

007 

2.00 

3.00 

1.00 

1.00 

4.00 

5.00 

2.00 

008 

1.00 

2.00 

2.00 

1.00 

1.00 

1.00 

2.00 

101 

2.00 

3.00 

h  2.00 

1.00 

4.00 

4.00 

2.00 

102 

1.00 

2.00 

2.00 

1.00 

3.00 

3.00 

4.00 

201 

2.00 

3.00 

1.00 

1.00 

5.00 

4.00 

1.00 

202 

4.00 

5.00 

4.00 

1.00 

4.00 

4.00 

3.00 

203 

3.00 

3.00 

3.00 

1.00 

5.00 

3.00 

3.00 

204 

2.00 

4.00 

1.00 

1.00 

5.00 

5.00 

1.00 

205 

3.00 

4.00 

2.00 

1.00 

4.00 

4.00 

3.00 

206 

2.00 

4.00 

1.00 

1.00 

5.00 

5.00 

5.00 

207 

3.00 

4.00 

2.00 

1.00 

4.00 

5.00 

1.00 

208 

2.00 

4.00 

1.00 

1.00 

5.00 

4.00 

4.00 

209 

2.00 

3.00 

3.00 

1.00 

5.00 

3.00 

2.00 

210 

3.00 

3.00 

5.00 

1.00 

2.00 

5.00 

5.00 

Table  17.  Post-Experiment  Questionnaire  -  Raw  data 

We  calculated  three  different  types  of  percentages,  as  shown  below,  to  help  reveal 
the  participants’  opinions: 
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overall  percentage 


30% 

20%/ 40% 

/  \ 

percentage  of  pilots  percentage  of  non-pilots 

Figure  4 1 .  Post  questionnaire  calculation 


Then  we  transferred  these  percentages  again  into  the  questionnaire  (see  Table  18). 
By  doing  it  this  way,  we  received  a  first  synopsis  of  the  summarized  subjective 


impression  of  all  participants. 


Question 

Answer-Op 

tions 

In  general,  how  do  you  judge  the 

Very 

Somewhat 

Border 

Somewhat 

Very  difficult 

difficulty  of  evaluating  the  current 
attitude  of  the  aircraft? 

easy 

easy 

line 

complicated 

0% 

20% 

40% 

30% 

10% 

(40%/0%) 

(30%/50%) 

(20%/40%) 

(5%/5%) 

(0%/0%) 

In  general,  how  do  you  judge  difficulty 

Very 

Somewhat 

Border 

Somewhat 

Very  difficult 

of  using  the  traditional  flight  instruments 
for  this  task? 

easy 

easy 

line 

complicated 

5% 

5% 

20% 

35% 

35% 

(10%/0%) 

(40%/0%) 

(30%/40%) 

(20%/50%) 

(0%/10%) 

In  general,  how  do  you  judge  the 

Very 

Somewhat 

Border 

Somewhat 

Very  difficult 

difficulty  of  using  the  WEBER-Box  for 
this  task? 

easy 

easy 

line 

complicated 

35% 

45% 

10% 

5% 

5% 

(30%/40%) 

(70%/20%) 

(0%/20%) 

(0%/10%) 

(0%/10%) 

In  general,  how  do  you  judge  the 

Very 

Somewhat 

Border 

Somewhat 

Very  poor 

benefits  of  color  coding  of  an  unusual 
attitude? 

good 

good 

line 

poor 

90% 

10% 

0% 

0% 

0% 

(80%/l00%) 

(20%/0%) 

(0%/0%) 

(0%/0%) 

(0%/0%) 

Did  you  feel  better  aware  of  the  spatial 
orientation  of  the  airplane  with  the 
WEBER-Box  in  comparison  with  the 

Much  less 

Somewhat 

less 

Border 

line 

Somewhat  more 

Much  more 

traditional  flight  instruments,  when  both 

5% 

5% 

15% 

40% 

35% 

are  without  color  coding  of  an  unusual 
attitude? 

(10%/0%) 

(0%/10%) 

(30%/0%) 

(50%/30%) 

(10%/60%) 

Did  you  feel  better  aware  of  the  spatial 
orientation  of  the  airplane  with  the 
WEBER-Box  in  comparison  with  the 

Much  less 

Somewhat 

less 

Border 

line 

Somewhat  more 

Much  more 

traditional  flight  instruments,  when  both 

5% 

0% 

25% 

45% 

25% 

are  with  color  coding  of  an  unusual 
attitude? 

(10%/0%) 

(0%/0%) 

(30%/20%) 

(50%/40%) 

(10%/40%) 

Did  having  the  WEBER-Box  increase  or 

Greatly 

Somewhat 

Unaffected 

Somewhat 

Greatly 

decrease  your  monitoring  demands  in 
comparison  with  the  traditional  flight 

decreased 

decreased 

20% 

increased 

increased 

instruments? 

20% 

40% 

10% 

10% 

(10%/30%) 

(70%/10%) 

(10%/30%) 

(10%/10%) 

(0%/20%) 

Table  18.  Results  of  the  post-experiment  questionnaire 
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This  subjective  data  provides  initial  evidence  about  the  preference  and  acceptance 
of  the  two  display  designs  and  the  opinion  of  the  participants  towards  the  option  of  a 
color-coded  feature  for  detecting  unusual  aircraft  attitudes. 

The  difficulty  using  the  traditional  display  design  forjudging  an  attitude  was  rated 
as  “border  line”  or  “somewhat  complicated”  by  70%  of  the  participants.  The  majority  of 
this  70%  are  non-pilots,  which  is  not  surprising  since  pilots  are  trained  with  this 
instrument.  Therefore,  the  pilots  considered  the  use  of  the  traditional  display  design 
relatively  easy.  On  the  other  hand,  80%  of  all  participants  rated  the  use  of  the  prototype 
display  design  as  “easy”  or  “somewhat  easy.”  Interestingly,  all  of  the  pilots  came  to  the 
same  conclusion.  This  gives  strong  evidence  that  pilots  as  well  as  non-pilots 
acknowledge  ease  of  use  of  the  prototype  design  and  felt  that  it  supports  their  task  of 
judging  different  attitudes. 

The  questionnaire  data  are  quite  clear  regarding  the  use  of  color-coding  for  the 
judgment  of  unusual  attitudes.  All  of  the  participants  acknowledged  the  advantage  of 
color-coding. 

Over  70%  of  the  participants  reported  that  they  were  more  aware  of  their  spatial 
orientation  with  the  prototype  design  (WEBER  Box),  independent  of  whether  it  was 
color-coded  or  not.  This  result  was  found  for  pilots  as  well  as  for  non-pilots. 
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VII.  CONCLUSIONS 


A.  OVERALL  ASSESSMENT  OF  THE  STATISTICAL  RESULTS 

The  main  question  for  an  overall  assessment  was  whether  a  significant  difference 
between  the  display  combinations  could  be  detennined. 

In  general,  the  effect  of  display  design  (traditional  or  WEBER-Box)  and  the  effect 
of  color  coding  was  significant.  Specifically  for  the  probability  of  correct  response, 
participants  using  the  WEBER-Box  reached  a  probability  of  correct  response  of  98% 
without  color  coding  and  100%  with  color  coding.  The  mean  time  also  decreased  when 
switching  from  the  traditional  display  to  the  WEBER-Box.  Adding  color  to  the  displays 
made  this  decrease  even  more  obvious.  Response  times  were  always  shorter  with  the 
WEBER-Box  than  with  the  traditional  display  design,  and  that  held  true  for  both  color 
coded  and  non  color  coded  display.  Therefore  the  WEBER-Box  was  better  than  the 
traditional  display  for  both  accuracy  of  determining  unusual  attitudes  and  speed  (response 
time). 

The  outcomes  also  must  be  considered  in  light  of  the  “pilot,  non-pilot”  variable. 
Since  both  pilots  and  non-pilots  participated,  one  has  to  take  into  consideration  that  pilots 
would  be  expected  to  perform  better.  They  are  not  only  familiar  with  the  traditional 
display  design,  but  they  are  also  trained  specifically  on  this  kind  of  task,  judging  an 
attitude  and  then  starting  a  recovery  operation  when  necessary.  This  also  enhances  their 
skills  to  read  and  interpret  flight  instruments.  Due  to  time  constraints  on  the  other  hand, 
there  was  very  little  time  to  familiarize  the  non-pilots  with  the  different  display  designs 
and  train  them  for  the  task  of  judging  an  aircraft  attitude. 

The  answers  in  the  post-experiment  questionnaire  confirmed  the  outcome  of  the 
statistical  analysis.  As  mentioned  in  the  previous  chapter,  there  is  agreement  between  the 
subjective  response  of  the  participants  in  the  post-experiment  questionnaire  and  the 
objective  data.  For  example,  both  pilots  and  non-pilots  felt  that  adding  color  to  each  of 
the  displays  increased  their  ability  for  spatial  awareness  and  spatial  orientation.  80%  of 
the  pilots  and  100%  of  the  non-pilots  judged  the  color-coding  to  be  beneficial. 
Comparing  the  traditional  display  design  with  the  prototype  display  design,  more  than 
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70%  of  the  participants  preferred  the  new  design,  independent  of  adding  color  or  not, 
compared  to  the  old  design.  This  result  gains  importance  because  60%  of  the  pilots 
judged  an  improved  awareness  with  the  prototype  design  as  “somewhat  more”  or  “much 
more”. 

A  detailed  look  into  the  data  shows  that  not  only  the  probability  of  correct 
answers  increased,  but  even  more  important,  the  time  required  to  judge  the  current 
attitude  decreased  significantly.  For  example  the  mean  response  times  changed  as 
follows: 

•  traditional  to  prototype:  from  1.92  sec  to  1.37  sec  (29%  reduction) 

•  without  color  to  color:  from  1.99  sec  to  1.30  sec  (36%  reduction) 

•  traditional  without  color  to  traditional  with  color:  from  2.35  sec  to  1.49  sec 
(37%  reduction) 

•  prototype  without  color  to  prototype  with  color:  from  1.62  sec  to  1.09  sec 
(33%  reduction) 

By  the  results  of  the  ANOVA  and  the  additional  /-tests,  one  can  see  that  the 
prototype  design  enabled  the  participants  to  judge  the  situation  faster  and  more 
accurately.  These  results  clearly  show  the  superiority  of  the  prototype  display  design 
(WEBER  Box). 

B.  DISCUSSION  AND  CONCLUSION 

To  understand  the  findings  of  this  study,  one  must  consider  the  purpose  of  the 
new  prototype  design.  It  was  built  to  support  pilots  or  operators  of  UAVs  in  extreme 
situations.  With  the  findings  of  the  research  questionnaire,  we  were  able  to  identify  a 
current  issue  of  the  UAV  community,  in  our  case  of  the  Predator  UAV.  That  is,  the 
WEBER-Box  would  help  in  the  recognition  of  the  current  aircraft  attitude,  facilitate 
recovery  when  an  unusual  attitude  existed,  and  help  to  prevent  misperception  of  an 
unusual  attitude.  Based  on  this  information,  we  developed  the  experiment  and  compared 
the  traditional  display  design  with  the  new  design  (WEBER  Box).  We  also  evaluated  the 
color  factor  and  its  two  levels,  with  or  without  color-coding. 
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In  the  experimental  task,  the  participants  judged  their  spatial  attitude  in  a  static 
environment.  Based  on  the  level  of  significance  of  a  =  0.05,  the  outcome  was 
remarkably  obvious.  Using  the  WEBER  Box,  performance  increased  in  both  accuracy 
and  response  time.  For  the  dependent  variable  “correct  response,”  using  the  prototype 
display  design  improved  perfonnance  by  105%.  When  using  the  color-coded  feature,  the 
same  105%  gain  of  performance  was  found.  The  results  are  even  more  significant  for  the 
dependent  variable  “mean  time  to  respond”  where  the  performance  improved  by  145% 
using  the  new  prototype  design. 

The  value  of  the  prototype  display  also  was  supported  by  the  post-experiment 
questionnaire  and  by  comments  and  suggestions  as  the  final  feedback  after  completing 
the  experiment.  Most  of  the  participants  stated  that  the  new  display  design  was  very 
helpful  in  judging  the  aircraft  attitude  task.  They  felt  an  improvement  in  their  spatial 
awareness  and  the  new  design  helped  them  to  decrease  the  time  for  correctly  judging 
their  attitude.  Thus,  the  prototype  display  design  made  it  easier  to  accomplish  their  task. 
By  adding  color  to  the  displays,  it  was  even  easier  to  judge  the  current  attitude  and  the 
percentage  of  correct  responses  increased  to  almost  100%  or  even  reached  a  level  of 
100%  in  the  case  of  the  new  design.  Likewise,  the  time  to  achieve  a  correct  response 
decreased  significantly  too. 

Therefore  the  combination  of  the  new  prototype  design  together  with  color-coding 
showed  a  clear  increase  of  performance  compared  to  the  traditional  display  design 
without  color  that  is  currently  being  used. 


C.  FUTURE  RESEARCH 

As  is  often  the  case  in  research,  in  addition  to  answering  some  questions,  other 
questions  and  suggestions  have  been  raised.  Most  of  them  fall  in  the  category  of 
suggested  design  improvements,  in  our  case  design  improvements  of  the  new  prototype 
design  in  combination  with  the  color  support.  These  can  be  summarized  as  follows: 

•  What  is  the  ideal  size  of  the  new  prototype  design?  Since  it  provides  a 
numerical  output  of  pitch  and  roll,  the  question  is  how  small  could  it  be  to 
still  fit  in  the  environment  of  a  ground  control  station  of  UAV  but  with  a 
font  size  still  large  enough  to  read  the  numbers. 
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•  Is  it  possible  to  decrease  the  time  for  judging  the  attitude  by  placing  the 
digital  readout  of  pitch  and  roll  to  a  more  central  location  in  the  display? 
Currently  there  is  still  an  eye  movement  necessary  from  the  animated 
aircraft  to  the  digital  pitch  and  roll  values,  since  it  is  not  possible  to  see  all 
of  the  information  with  one  view. 

•  Would  it  be  better  to  change  the  color  of  the  entire  aircraft  avatar  instead 
of  the  output  of  pitch  and  roll  digital  display,  when  reaching  an  unusual 
attitude? 

•  Can  the  output  pitch  and  roll  be  represented  in  a  different  way  so  that  the 
aircraft  avatar  and  the  output  can  be  seen  at  the  same  time?  A  possible 
solution  to  this  question  is,  giving  the  new  prototype  display  design  a 
background  like  a  grid.  The  aircraft  avatar  would  “fly”  on  top  of  the  grid 
and  pitch  and  roll  as  additional  numerical  output  could  be  displayed 
sideways  to  this  grid. 
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APPENDIX  A.  RESEARCH  QUESTIONNAIRE 


Naval  Postgraduate  School,  LTC  Thomas  Zirkelbach,  German  Army  Jan  2007 

Background  Information 

Spatial  awareness  and  the  skills  of  an  operator  to  transition  from  the  current  spatial 
attitude  into  the  necessary  spatial  orientation  and  position  mainly  depend  on  human 
performance  in  spatial  orientation  tasks.  Degraded  spatial  awareness,  particularly  in 
extreme  flight  situations,  may  lead  to  loss  of  equipment  and  to  lower  operational 
effectiveness.  How  to  maintain  and  improve  spatial  awareness  is  one  of  the  major  issues 
in  complex  3D  environments.  In  particular,  orientation  tasks  in  3D  environments  with 
many  degrees  of  freedom  are  very  difficult  to  accomplish. 

Based  on  these  findings  Maj  Axel  Weber  (GE)  has  developed  in  his  thesis  “Application 
of  Avatars  in  Display  Design  to  support  Spatial  Awareness  in  Extreme  Flight  Situations” 
a  prototype  display  design  that  helps  to  recognize  and  recover  from  an  unusual  attitude 
faster  and  more  accurate  compared  to  a  traditional  display  design. 


Screenshot  of  the  new  prototype  display  design,  called  WEBER-Box 


I  am  expanding  this  work  researching  with  my  thesis  “Pictorial  Display  Design  to 
enhance  Spatial  Awareness  of  Operators  in  Umnanned  Aviation”  possible  applications  of 
the  WEBER-Box  for  UAVs. 

Therefore  I  am  currently  looking  for  UAV  incidents  which  will  be  the  initial  position  for 
an  experiment  that  should  have  a  realistic  background.  Therefore  I  am  assuming  that 
operations  of  large  UAVs  are  normally  not  flown  in  unusual  attitude,  but  nevertheless 
occasionally  it  occurs.  One  basic  question  is  “How  often  do  the  operators  get  unintended 
into  an  unusual  attitude?”  Thus  I  have  elicited  and  focused  through  my  research  on  two 
general  approaches. 

Approach  1 :  Recovery  from  an  unusual  attitude 

Currently  available  literature  states  that  incidents  occur  if  the  data  link  is  interrupted,  will 
be  reestablished  and  a  recovery  of  the  UAV  is  still  not  possible.  I  am  also  thinking  of  a 
different  situation  where  there  are  no  problems  with  the  data-link  or  the  GPS-Signal  but 
the  internal  computer  which  is  responsible  for  the  stabilization  system  fails  and  the  UAV 
e.  g.  Predator  has  to  be  recovered  manually  by  the  operator  in  the  Ground  Control 
Station.  In  such  a  case  I  assume  the  UAV  will  be  in  an  unusual  attitude  before  the 
recovery  even  starts. 
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Approach  2:  Launch  and  Recovery 

With  this  approach  I  focus  on  the  topic  how  the  WEBER-Box  could  support  recovery 
operations. 


Naval  Postgraduate  School,  LTC  Thomas  Zirkelbach,  German  Army  Jan  2007 

Questionnaire 


1)  We  have  a  prototype  display  design  that  helps  to  recognize  and  recover  from  an 
unusual  attitude. 

a)  Can  you  think  of  a  situation  where  it  can  be  used? 

2)  a)  Have  you  experienced  situations  of  unusual  attitude? 

b)  Which  UAV  has  been  involved? 

c)  What  was  the  cause  for  this  unusual  attitude? 

d)  How  did  you  recover  it,  autonomously  or  manually? 

e)  If  manually,  which  instruments  did  you  use? 

f)  Have  you  been  successful? 

3)  a)  Have  you  experienced  situations  of  unusual  attitude  when  handing  over  a  UAV 

from  one  Ground  Station  to  another? 

b)  If  yes,  which  UAV  has  been  involved? 

c)  Please  describe: 

4)  a)  Have  you  experienced  situations  of  unusual  attitude  after  an  interrupted  data  link 

was  restored? 

b)  If  yes,  which  UAV  has  been  involved? 

c)  Please  describe: 

5)  Assume  the  following  scenario:  a  Predator  has  no  data-link  interruption  and  the  GPS 
signal  works,  but  its  internal  computer  responsible  for  its  autonomous  flying  fails. 

a)  Have  you  ever  experienced  a  situation  like  this? 

b)  Does  the  operator  take  over  in  remote  control? 

c)  Did  this  situation  cause  an  unusual  attitude? 

6)  Can  you  launch  and  recover  in  instrument  conditions? 
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7)  If  you  had  to  accomplish  evasive  maneuvers  to  escape  anti-aircraft  fire, 

a)  Would  you  do  this  in  remote  control? 

b)  Which  flight  instruments  would  you  be  using? 

c)  How  difficult  would  this  be? 
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APPENDIX  B.  IRB  APPLICATION 


Michael  E.  McCauley,  Ph.D. 
Operations  Research  Department 
Glasgow  Hall 
Naval  Postgraduate  School 
Monterey,  California  93943 


831-656-2191 
DSN:  756-2191 
Fax:  831-656-2595 
memccaul@nps.edu 


To:  Protection  of  Human  Subjects  Committee 


Subject:  Application  for  Human  Subjects  Review  for 

Pictorial  Display  Design  to  enhance  Spatial  Awareness  of  Operators  in 
Unmanned  Aviation 


1 .  Attached  is  a  set  of  documents  outlining  a  proposed  experiment  to  be  conducted  over 
the  three  month  to  support  the  thesis  of  LTC  Thomas  Zirkelbach,  GE. 

2.  We  are  requesting  approval  of  the  described  experimental  protocol.  An  experimental 
outline  is  included  for  your  reference  that  describes  the  methods  and  measures  we 
plan  to  use. 

3.  We  include  the  consent  forms,  privacy  act  statements,  all  materials  and  forms  that  a 
subject  will  read  or  fill-out,  and  the  debriefing  forms  (if  applicable)  we  will  be  using 
in  the  experiment. 

4.  We  understand  that  any  modifications  to  the  protocol  or  instruments/measures  will 
require  submission  of  updated  IRB  paperwork  and  possible  re-review.  Similarly,  we 
understand  that  any  untoward  event  or  injury  that  involves  a  research  participant  will 
be  reported  immediately  to  the  IRB  Chair  and  NPS  Dean  of  Research. 


Michael  E.  McCauley 
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APPLICATION  FOR 
HUMAN  SUBJECTS  REVIEW 


(HSR) 


NPS  IRB  NUMBER 
RSPO) _ 


(to  be  assigned  by 


PRINCIPAL  INVESTIGATOR(S)  (Full  Name,  Code,  Telephone) 

Michael  E.  McCauley,  Ph.D.,  Monterey,  CA  93943,  Phone:  831  656-2191 
(Thesis  of  LTC  Thomas  Zirkelbach,  GE) 


TITLE  OF  EXPERIMENT 

Does  an  outside-in  aircraft  animation  display  help  UAV  operators  to  accurately  perceive 
unusual  attitudes? 


APPROVAL  REQUESTED  [X]  New  [  ]  Renewal  [  ]  Amendment 


LEVEL  OF  RISK  [  ]  Exempt  [X]  Minimal  [  ]  More  than  Minimal 
Justification:  Since  there  is  no  wide  field  of  view  (FOV)  terrain  display,  simulator  sickness 
is  extremely  unlikely.  A  prior  study  using  this  prototype  display  found  zero  incidence  of 
simulator  sickness. 


WORK  WILL  BE  DONE  IN  (Site/Bldg/Rm) 

NPS,  Watkins  Hall,  RM  212B 

ESTIMATED  NUMBER  OF  DAYS  TO 
COMPLETE 

30 

MAXIMUM  NUMBER  OF  SUBJECTS 

50 

ESTIMATED  LENGTH  OF  EACH 
SUBJECT’S  PARTICIPATION 

2  hours  (1  hour  in  flight  simulation) 

SPECIAL  POPULATIONS  THAT  WILL  BE  USED  AS  SUBJECTS 


[  ]  Subordinates  [  ]  Minors  [X]  NPS  Students  [  ]  Special  Needs  (e.g.  Pregnant  women) 

Specify  safeguards  to  avoid  undue  influence  and  protect  subject’s  rights: 

Participant  Consent  Form  &  Minimal  Risk  Statement 


SCIENTIFIC  MERIT  REVIEW  (Check  all  that  apply) 

[  ]  This  research  is  part  of  a  funded  project  (Job  Order  Number  ) 

[X]  This  research  is  a  student  thesis  (Attach  a  copy  of  the  approved  thesis  proposal) 
j  ]  Other  (Attach  a  complete  research  proposal) 


OUTSIDE  COOPERATING  INVESTIGATORS  AND  AGENCIES 

None 

|  ]  A  copy  of  the  cooperating  institution’s  HSR  decision  is  attached. 
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DESCRIPTION  OF  RESEARCH  (attach  additional  sheet  if  needed). 

Methodology  attached 

I  have  read  and  understand  NPS  Notice  on  the  Protection  of  Human  Subjects.  If  there  are  any 
changes  in  any  of  the  above  information  or  any  changes  to  the  attached  Protocol,  Consent 
Form,  or  Debriefing  Statement,  I  will  suspend  the  experiment  until  I  obtain  new  IRB  approval. 

SIGNATURE  DATE 
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Methodology 


Experiment  Goals  and  Purpose 

This  research  will  investigate  the  effectiveness  of  a  prototype  display  for  pilot/operator 
recognition  of  aircraft/UAV  attitude. 

Research  Questions 

Will  the  new  display  design  help  operators  to  recognize  aircraft  attitude  accurately? 

Will  the  new  design  help  in  discriminating  between  a  normal  and  unusual  attitude? 

Will  the  new  design  help  prevent  misperception  of  an  unusual  attitude? 

Will  the  new  design  reduce  errors  in  recovery  from  unusual  attitudes? 

Will  the  new  design  support  the  substitution  of  operators  for  trained  pilots  in  unmanned 
aviation  tasks? 

What  will  be  the  level  of  user  acceptance  of  operators  and  pilots  regarding  the  new 
design? 

Participants 

The  participants  will  be  randomly  assigned  on  a  volunteer  basis  from  students  and  faculty 
of  NPS.  The  study  will  involve  at  the  most  50  participants.  It  is  intended  to  split  them 
equally  into  25  pilots  and  25  non-pilots. 

Apparatus 

The  experiment  will  take  place  at  the  Naval  Postgraduate  School,  Monterey,  California, 
Watkins  Building,  room  212B.  Two  workstations  will  be  used.  Workstation  1  is  for 
instruction  and  training  while  Workstation  2  will  be  established  for  the  underlying 
experiment. 

Workstation  1; 

The  computer-generated  images  will  be  displayed  on  a  19”  computer  monitor  with  a  1024 
x  768  pixel  resolution.  The  pictures  of  the  flight  situations  were  generated  by  screenshots 
of  the  commercial  flight  simulator  software  X-Plane™,  version  7.13  and  by  screenshots  of 
the  WEBER-Box  program.  The  pictures  were  stored  in  a  PowerPoint™  slideshow  to  ensure 
that  every  participant  will  see  the  different  aircraft  attitudes  in  the  same  order.  This 
slideshow  will  be  used  to  explain  the  differences  between  a  usual  and  unusual  attitude.  The 
simulation  of  the  flight  situations  also  will  be  generated  by  X-Plane™  and  by  the  WEBER- 
Box  program.  The  predefined  simulations  of  usual  and  unusual  attitude  will  be  stored  in  X- 
Plane™  to  help  the  participants  to  gain  a  better  understanding  of  this  complex  situation  in  a 
dynamic  environment. 

Workstation  2; 

The  computer-generated  images  will  be  displayed  on  two  19”  computer  monitors  with  a 
1024  x  768  pixel  resolution.  The  two  screens  will  be  located  on  top  of  each  other,  similar  to 
the  ground  station  of  a  Predator  UAV.  The  simulation  of  the  flight  situations  will  be 
generated  by  the  commercial  flight  simulator  software  X-Plane™,  version  7.13  and  by  the 
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WEBER-Box  program.  The  predefined  simulations  of  usual  and  unusual  attitude  will  be 
stored  in  X-Plane™  to  ensure  that  every  participant  will  see  the  different  attitudes  in  the 
same  order. 

Procedures 

The  overall  experiment  will  be  subdivided  into  four  phases. 

Phase  one  —  Preparation: 

The  participant  will  be  asked  to  fill  out  the  necessary  paperwork,  including  a  consent  form, 
a  privacy  act  statement,  a  minimal-risk  information  form,  and  an  initial/  demographic 
questionnaire. 

Phase  two  -  Introduction  and  Training 

The  participant  will  be  introduced  to  the  experiment,  receiving  information  about  the  UAV, 
the  task  as  an  operator  of  the  UAV,  and  details  about  usual  and  unusual  attitudes.  The 
participant  also  will  watch  a  10-minute  introduction  video  of  usual  and  unusual  attitudes  as 
displayed  by  the  traditional  flight  instruments  and  by  the  WEBER-Box.  This  pre- 
experimental  period,  phase  one  and  two,  will  take  approximately  25  minutes. 

Phase  three  —  Experiment 

The  participant  will  start  with  the  first  experimental  sub-task.  Each  of  his  or  her  14  trials 
will  be  scheduled  for  120  seconds.  The  first  two  trials  will  be  practice  trials.  The 
experimenter  will  start  the  prepared  simulation  of  several  flight  situations  that  end  in  a 
usual  or  unusual  attitude  and,  after  120  seconds,  the  simulation  will  stop  with  a  picture  of 
flight  instruments  in  a  certain  flight  situation.  The  participant  will  judge  the  “frozen” 
situation  as  a  usual  or  unusual  attitude.  Then,  the  experimenter  will  initiate  the  next  trial 
until  the  participant  has  finished  twelve  trials,  plus  two  test  trials. 

After  accomplishing  the  first  block  of  experiments,  the  participant  will  take  a  5-minute 
break.  Then  the  participant  will  start  the  second  experimental  task.  All  the  participants  will 
be  exposed  to  all  of  the  instrument  setups  and  all  of  the  sub-experiments  (a  within-subjects 
design).  Each  participant  will  be  presented  with  seven  trails  of  usual  attitude  with 
traditional  design  and  seven  trails  of  unusual  attitude  with  the  traditional  design.  Then  the 
same  procedure  will  apply  with  the  new  display  design,  the  WEBER  Box. 

Phase  four  -  Questionnaire 

When  finished,  the  participant  will  be  asked  to  fill  out  the  post-experimental  questionnaire. 
This  completes  the  experimental  session. 
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APPENDIX  C.  IRB  APPROVAL 


Naval  Postgraduate  School 

Institutional  Review  Board  (IRB) 


22-Feb-07 


From: 

To: 


LT  Brent  Olde,  Ph.D. 

Research  Professor  Michael  McCauley 


Subject:  YOUR  PROJECT:  PICTORIAL  DISPLAY  DESIGN  TO  ENHANCE 

SPATIAL  AWARENESS  OF  OPERATORS  IN  UNMANNED 
AVIATION 

1.  The  NPS  IRB  is  pleased  to  inform  you  that  the  NPS  Institutional  Review 
Board  has  approved  your  project  (NPS  IRB#  NPS20070042). 

2.  The  NPS  IRB  was  originally  certified  by  BUMED  on  26  July  2002  and 
has  been  re-certified  until  30  March  2007. 

3.  This  approval  is  valid  for  one  year  from  (his  date.  Please  submit  a  copy  of 
all  records  and  consent  forms  to  the  Research  and  Sponsored  Programs 
Office  (Laura  Ann  Small,  Halligan  Hall,  Room  201 B)  at  the  conclusion  of 
this  project. 

4.  If  your  protocol  changes  at  any  lime,  you  will  need  to  resubmit  your 
project  proposal  to  the  NPS  IRB. 


Sincerely, 


Lt  Brent  Olde,  Ph  D. 

Chair 

NPS  Institutional  Review  Board 
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APPENDIX  D.  INTRODUCTION  /  TASK  DESCRIPTION 


Pictorial  Display  to  Enhance  Spatial 


Awareness  of  Operators  in 
Unmanned  Aviation 

! si 


LTC  Thomas  Zirkelbach 
German  Army 


Thank  you  participating  in  this 
experiment.  Without  your  support  and 
your  dedication  I  would  never  be  able 
to  accomplish  my  mission  finishing 
this  experiment  and  therefore  my 
thesis. 

Thank  you  very  much! 


Note:  To  continue,  please  push  the  return  key. 
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Thesis  Experiment 
Phase  1:  Pre-Paperwork  - 


Thesis  Experiment 
-  Phase  1:  Pre-Paperwork 


From  the  additional  papers  provided  to  you, 
please  read  and  sign  only  the  Initial 
Questionnaire  and  leave  the  field  Participant  ID 


When  you  are  done,  please  continue  with  the  presentation.  7 


r  raiV ..  Thesis  Experiment 

-  Phase  2:  Introduction  - 


Problem  statement 

In  manned  as  well  as  in  unmanned  aviation  the 
pilot/  operator  is  sometimes  faced  with  an 
unusual  aircraft  attitude.  Often,  his  task  is  to 
recover  from  this  unusual  attitude. 

In  relation  to  the  speed  and /  or  altitude  it  is  very 
important  to  recover  quickly  and  accurately. 

Depending  on  your  mission  and  current  task, 
quick  recovery  may  be  important. 
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Thesis  Experiment 

V 

-  Phase  2:  Introduction  - 

It  provides  an  ,  meaning  a  view  from 

the  inside  of  the  aircraft  to  the  outside  world. 


If  aircraft  reaches  an  unusual  attitude  pitch  and/  or  roll  will  appear 
in  yellow  in  the  left  bottom  corner  of  the  display 
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Thesis  Experiment 

V 

-  Phase  2:  Introduction  - 

Instruments  -  New  Prototype  Design,  the  WEBER-Box 
A  new  prototype  display  called  the  WEBER-Box 
has  recently  been  designed. 

As  opposed  to  the  traditional  design,  it  provides 
an  ,  meaning  a  view  from  the 

outside  world  into  the  aircraft. 


Thesis  Experiment 

V 

-  Phase  2:  Introduction  - 

Scale  of  the  new  Prototype  Design 


Information 

pitch 

bank  angle  x 


16 


IftH 

Thesis  Experiment 

V 

-  Phase  2:  Introduction  - 

If  aircraft  reaches  an  unusual  attitude  the  color  of 
pitch/  hank  angle  will  switch  to  yellow 
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Thesis  Experiment 

Mm 

-  Phase  2:  Training  - 

jCX 


II  2f€7:  H 

Is  this  a  usual I  unusual  attitude? 

,  since  level  flight. 

21 


V 


Thesis  Experiment 
■  Phase  2:  Training  - 


-j  m 
"t  »- 


_Cl_ 


t: 

2.3(0 


Is  this  a  usual/  unusual  attitude? 

,  since  pitch  >  25°  nose  up. 


r  rb§L  Thesis  Experiment 

-  Phase  2:  Training  - 


Is  this  a  usual I  unusual  attitude? 


,  since  pitch  >  10°  nose  down. 

23 
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Thesis  Experiment 

Mm 

-  Phase  2:  Training  - 

,1*1, 

^  p= 

L. 

i  jdi 

Is  this  a  usual I  unusual  attitude? 

,  since  pitch  <  25°  nose  up  and 

bank  angle  <  45°. 


rcsIL  Thesis  Experiment 

-  Phase  2:  Training  - 


Is  this  a  usual  or  unusual  attitude? 

,  since  bank  angle  <  45°. 

25 


Thesis  Experiment 

V 

-  Phase  2:  Training  - 

Is  this  an  usual/  unusual  attitude? 

,  since  bank  angle  >  45°. 

26 
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Thesis  Experiment 
■  Phase  2:  Training  - 


Is  this  a  usual  or  unusual  attitude? 
,  since  pitch  >  25°  nose  up. 


rcsIL  Thesis  Experiment 

■  Phase  2:  Training  - 


Is  this  a  usual  or  unusual  attitude? 

,  since  bank  angle  <  45°. 
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Thesis  Experiment 

V 

-  Phase  2:  Training  - 

Is  this  an  usual/  unusual  attitude? 

,  since  pitch  <  25°  nose  up  and 

bank  angle  <  45°. 

29 
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Thesis  Experiment 

mm 

-  Phase  2:  Training  - 

Is  this  an  usual/  unusual  attitude? 

,  since  level  flight. 

3D 


rcsIL  Thesis  Experiment 

■  Phase  2:  Training  - 


1/ 


Is  this  an  usual/  unusual  attitude? 
,  since  bank  angle  >  45°. 


Thesis  Experiment 

V 

-  Phase  2:  Training  - 

Is  this  an  usual/  unusual  attitude? 

,  since  pitch  >  10°  nose  down. 

32 
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V 


Thesis  Experiment 
-  Phase  3:  Experiment  - 


The  Task 

You  are  currently  assigned  as  a  pilot  of  the  UAV 
MQ-1  Predator. 


Thesis  Experiment 
■  Phase  3:  Experiment  - 

Features  of  the  Predator  “For  vour  information” 

Primaiy  Function: 

Armed  reconnaissance,  airborne  surveillance,  target 
acquisition 

Contractor: 

General  Atomics  Aeronautical  Systems  Incorporated 

Power  Plant: 

Rotax  914  four  cylinder  engine  with  101  horsepower 

Length: 

27  feet  (8.22  meters) 

Height: 

6.9  feet  (2.1  meters) 

Weight: 

1,130  pounds  (512  Kg)  empty,  maximum  takeoff  weight 
2,250  pounds  (1,020  kg) 

Wingspan: 

48.7  feet  (14.8  meters) 

Speed: 

Cruise  speed  around  84  rnph  (70  knots),  up  to  135  rnph 

Range: 

up  to  400  nautical  miles  (454  miles) 

Ceiling: 

up  to  25,000  feet  (7,620  meters) 

Fuel  Capacity: 

665  pounds  (100  gallons) 

Payload: 

450  pounds  (204  kg) 

Armament: 

two  laser-guided  AGM-1 14  Hellfire  anti-tank  missiles  34 

Thesis  Experiment 

Kflfl 

-  Phase  3:  Experiment  - 

General  information 

The  MQ-1  Predator  is  a  medium-altitude,  long-endurance, 
remotely  piloted  aircraft.  It’s  primary  mission  is 
interdiction  and  conducting  armed  reconnaissance 
against  critical,  perishable  targets.  When  it  is  not  actively 
pursuing  its  primary  mission,  it  acts  as  the  Joint  Forces 
Air  Component  Commander-owned  theater  asset  for 
reconnaissance,  surveillance  and  target  acquisition  in 
support  of  the  Joint  Forces  Commander. 

The  basic  crew  is  one  pilot  and  two  sensor  operators. 

They  fly  the  aircraft  from  inside  the  ground  control  station 
via  a  line-of-sight  data  link  or  a  satellite  data  link  for 
beyond  line-of-sight  flight. 

36 
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Thesis  Experiment 
-  Phase  3:  Experiment  ■ 


General  information 

The  aircraft  is  equipped  with  a  color  camera  in  the  nose 
(generally  used  by  the  pilot  for  flight  control),  a  day 
variable-aperture  TV  camera,  a  variable-aperture  infrared 
(IR)  camera  (for  low  light/night),  and  a  synthetic  aperture 
radar  (SAR)  for  looking  through  smoke,  clouds  or  haze. 
The  cameras  produce  full  motion  video  while  the  SAR 
produces  still-frame  radar  images. 

For  this  experiment  there  will  be  a  slight  change.  The  nose 
camera  will  be  switched  off  since  your  main  focus  will  be 
on  your  instruments.  This  would  be  equivalent  to  flying  in 
conditions  of  poor  visibility. 


A, 

Thesis  Experiment 

•  Phase  3:  Experiment  - 

The  Task 


Your  duty  station  is  the 
Ground  Control  Station, 
located  in  Monterey,  at  the 
Naval  Postgraduate  School 

To  gain  a  better 
understanding  about  the 
work  of  an  operator  of  a 
Predator,  please  view  this 
video  by  pressing  I 


rail ..  Thesis  Experiment 

-  Phase  3:  Experiment  - 


Experiment  Goals 

The  goal  of  this  experiment  is  to  test  different  flight 
instruments  (Traditional  HUD  and  new  prototype  design 
WEBER-Box)  with  different  characteristics  (color  and 
non-color)  for  supporting  judgment  of  aircraft  attitude. 


Overall  Experiment  Schedule 

You  will  conduct  four  sets  of  trials.  The  order  in  which 
you  will  experience  the  four  sets  is  assigned  randomly. 
The  only  difference  between  the  four  sets  is  which  set  of 
flight  instruments  is  used  and  if  it  is  color-coded  or  not. 


95 


Thesis  Experiment 
-  Phase  3:  Experiment  ■ 
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APPENDIX  E.  PARTICIPANT  CONSENT  FORM  &  MINIMAL 

RISK  STATEMENT 


Naval  Postgraduate  School 

Participant  Consent  Form  & 
Minimal  Risk  Consent  Statement 


Introduction.  You  are  invited  to  participate  in  a  study  entitled  Pictorial  Display  Design 
to  enhance  Spatial  Awareness  of  Operators  in  Unmanned  Aviation  being  conducted  by 
the  Naval  Postgraduate  School  MOVES  Institute. 

Procedures.  If  I  agree  to  participate  in  this  study,  I  understand  that  I  will  be  provided  with 
an  explanation  of  the  purposes  of  the  research,  a  description  of  the  procedures  to  be  used, 
identification  of  any  experimental  procedures,  and  the  expected  duration  of  my 
participation. 

Synopsis :  There  will  be  two  sessions:  (1)  a  30  minute  test  phase  with  traditional 
flight  instruments  and  (2)  a  30  minute  test  phase  with  the  new  prototype  display  design, 
during  which  you  will  be  expected  to  accomplish  a  number  of  tasks  related  to  orientation 
during  flight  operations. 

Risks  and  Benefits.  I  understand  that  this  project  does  not  involve  greater  than  minimal 
risk  and  involves  no  known  reasonably  foreseeable  risks  or  hazards  greater  than  those 
encountered  in  everyday  life.  I  understand  that  there  is  a  very  small  chance  that  susceptible 
participants  could  experience  symptoms  of  simulator  sickness  such  as  “queasy”  stomach.  I 
agree  that,  if  I  do  experience  such  symptoms,  I  will  remain  in  the  testing  area  until  they 
have  subsided.  I  have  also  been  informed  of  any  benefits  to  myself  or  to  others  that  may 
reasonably  be  expected  as  a  result  of  this  research. 

Compensation.  I  understand  that  no  tangible  reward  will  be  given.  I  understand  that  a 
copy  of  the  research  results  will  be  available  at  the  conclusion  of  the  experiment. 

Confidentiality  &  Privacy  Act.  I  understand  that  all  records  of  this  study  will  be  kept 
confidential  and  that  my  privacy  will  be  safeguarded.  No  information  will  be  publicly 
accessible  which  could  identify  me  as  a  participant,  and  I  will  be  identified  only  as  a  code 
number  on  all  research  forms.  I  understand  that  records  of  my  participation  will  be 
maintained  by  NPS  for  five  years,  after  which  they  will  be  destroyed. 

Voluntary  Nature  of  the  Study.  I  understand  that  my  participation  is  strictly  voluntary, 
and  if  I  agree  to  participate,  I  am  free  to  withdraw  at  any  time  without  prejudice. 

Points  of  Contact.  I  understand  that  if  I  have  any  questions  or  comments  regarding  this 
project  upon  the  completion  of  my  participation,  I  should  contact  the  Principal  Investigator, 
Dr.  Michael  E.  McCauley,  656-2191,  memccaul@nps.edu.  Any  medical  questions  should 
be  addressed  to  LTC  Eric  Morgan,  MC,  USA,  (CO,  POM  Medical  Clinic),  (831)  242-7550, 
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eric.morgan@nw.amedd.army.mil.  Any  other  questions  or  concerns  may  be  addressed  to 
the  IRB  Chair,  LT  Brent  Olde,  656-3807,  baolde@nps.edu. 

Statement  of  Consent.  I  have  read  and  understand  the  above  information.  I  have  asked 
all  questions  and  have  had  my  questions  answered.  I  agree  to  participate  in  this  study.  I 
will  be  provided  with  a  copy  of  this  form  for  my  records. 


Participant’s  Signature 


Date 


Researcher’s  Signature 


Date 
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APPENDIX  F.  PRIVACY  ACT  STATEMENT 


NA  VAL  POSTGRADUATE  SCHOOL,  MONTEREY,  CA  93943 
PRIVA  CYA  CT  ST  A  TEMENT 


1 .  Purpose:  Spatial  cognition  data  will  be  collected  to  enhance  knowledge,  and  to 
develop  tests,  procedures,  and  equipment  to  improve  the  development  of  Virtual 
Environments. 

2.  Use:  Spatial  cognition  data  will  be  used  for  statistical  analysis  by  the  Departments  of 
the  Navy  and  Defense,  and  other  U.S.  Government  agencies,  provided  this  use  is 
compatible  with  the  purpose  for  which  the  information  was  collected.  Use  of  the 
information  may  be  granted  to  legitimate  non-government  agencies  or  individuals  by 
the  Naval  Postgraduate  School  in  accordance  with  the  provisions  of  the  Freedom  of 
Information  Act. 

3.  Disclosure/Confidentiality: 

a.  I  have  been  assured  that  my  privacy  will  be  safeguarded.  I  will  be  assigned  a 
control  or  code  number  which  thereafter  will  be  the  only  identifying  entry  on 
any  of  the  research  records.  The  Principal  Investigator  will  maintain  the  cross- 
reference  between  name  and  control  number.  It  will  be  decoded  only  when 
beneficial  to  me  or  if  some  circumstances,  which  is  not  apparent  at  this  time, 
would  make  it  clear  that  decoding  would  enhance  the  value  of  the  research  data. 
In  all  cases,  the  provisions  of  the  Privacy  Act  Statement  will  be  honored. 

b.  I  understand  that  a  record  of  the  infonnation  contained  in  this  Consent  Statement 
or  derived  from  the  experiment  described  herein  will  be  retained  permanently  at 
the  Naval  Postgraduate  School  or  by  higher  authority.  I  voluntarily  agree  to  its 
disclosure  to  agencies  or  individuals  indicated  in  paragraph  3  and  I  have  been 
informed  that  failure  to  agree  to  such  disclosure  may  negate  the  purpose  for 
which  the  experiment  was  conducted. 

c.  I  also  understand  that  disclosure  of  the  requested  information,  including  my 
Social  Security  Number,  is  voluntary. 


Name,  Grade/Rank  (if  applicable) 
[Please  print] 


Signature  of  Volunteer 


101 


Date 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


102 


APPENDIX  G.  INITIAL  QUESTIONNAIRE 


Date: 


Initial  Questionnaire 


Age 

Gender 

Male  □  Female  dl 

Are  you  experienced  in  aviation  in  any 
regard  (incl.  playing  Flight  Simulation 
Games)? 


YES  □ 


NO  □ 


If  you  answered  the  last  question  with 
YES,  please  proceed  HERE. _ 


Are  you  an  aviator/pilot? 

YES  □  NO  □ 

Are  you  familiar  with  the  basic  set  of 
flight  instruments? 

YES  □  NO  □ 

Are  you  familiar  with  the  basic  aircraft 
flight  controls? 

YES  □  NO  □ 

Did  you  operate  any  of  these  aircraft?: 
Fixed-Wing  (Jet/Propeller) 

Rotor-Wing  Aircraft 

UAV 

Fixed  □  Rotor  dl 

Jet  □  Propeller  dl 

UAV  □ 

How  many  flight  hours  do  you  have? 

Hours 

How  many  hours  in  a  flight  simulator  do 
you  have? 

Hours 

How  long  ago  was  your  last  flight? 

Years  Month(s) 

How  long  ago  was  your  last  use  of  a 
flight  simulator? 

Years  Month(s) 

Are  you  trained  in  procedures  of 
recovery  from  unusual  attitude? 

YES  □  NO  □ 
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Leave  blank! 


Participant  ID 


If  you  are  an  active/former  aviator/pilot, 
please  proceed  HERE: _ 


Have  you  ever  experienced  any  kind  of 
Spatial  Disorientation? 

YES  □  NO  □ 

More  than  once? 

YES  □  NO  □ 

Briefly  describe  the  situation/s 

PLEASE  STOP  HERE! 
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APPENDIX  H.  POST-EXPERIMENT  QUESTIONNAIRE 


Post-Experiment  Questionnaire 


Leave  blank! 


Question 

Answer-Opi 

tions 

In  general,  how  do  you  judge  the 
difficulty  of  evaluating  the  current 
attitude  of  the  aircraft? 

Very 

easy 

Somewhat 

easy 

Border 

line 

Somewhat 

complicated 

Very  difficult 

In  general,  how  do  you  judge  difficulty 
of  using  the  traditional  flight  instruments 
for  this  task? 

Very 

easy 

Somewhat 

easy 

Border 

line 

Somewhat 

complicated 

Very  difficult 

In  general,  how  do  you  judge  the 
difficulty  of  using  the  WEBER-Box  for 
this  task? 

Very 

easy 

Somewhat 

easy 

Border 

line 

Somewhat 

complicated 

Very  difficult 

In  general,  how  do  you  judge  the 
benefits  of  color  coding  of  an  unusual 
attitude? 

Very 

good 

Somewhat 

good 

Border 

line 

Somewhat 

poor 

Very  poor 

Did  you  feel  better  aware  of  the  spatial 
orientation  of  the  airplane  with  the 
WEBER-Box  in  comparison  with  the 
traditional  flight  instruments,  when  both 
are  without  color  coding  of  an  unusual 
attitude? 

Much  less 

Somewhat 

less 

Border 

line 

Somewhat  more 

Much  more 

Did  you  feel  better  aware  of  the  spatial 
orientation  of  the  airplane  with  the 
WEBER-Box  in  comparison  with  the 
traditional  flight  instruments,  when  both 
are  with  color  coding  of  an  unusual 
attitude? 

Much  less 

Somewhat 

less 

Border 

line 

Somewhat  more 

Much  more 

Did  having  the  WEBER-Box  increase  or 
decrease  your  monitoring  demands  in 
comparison  with  the  traditional  flight 
instruments? 

Greatly 

decreased 

Somewhat 

decreased 

Unaffected 

Somewhat 

increased 

Greatly 

increased 
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APPENDIX  I.  COMMENTS/  SUGGESTIONS/  OPINIONS 


In  General: 


Experiment  Tasks: 


Traditional  Instruments: 


WEBER-Box: 
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APPENDIX  J.  ORDER  OF  TRIALS 


Order  of  Trials 


Project  Study  02 


Participant 

Order 

1. 

SOI 

NP 

A 

B 

C 

D 

2. 

S10 

P 

B 

C 

D 

A 

3. 

NP 

C 

D 

A 

B 

Experiment 


Participant 

Order 

1. 

201 

NP 

A 

B 

C 

D 

2. 

101 

P 

B 

C 

D 

A 

3. 

NP 

C 

D 

A 

B 

4. 

P 

D 

A 

B 

C 

5. 

mm 

NP 

A 

B 

C 

D 

6. 

P 

B 

C 

D 

A 

7. 

003 

P 

C 

D 

A 

B 

8. 

004 

P 

D 

A 

B 

C 

9. 

005 

P 

A 

B 

C 

D 

10. 

204 

NP 

B 

C 

D 

A 

11. 

205 

NP 

C 

D 

A 

B 

12. 

■■mm 

P 

D 

A 

B 

C 

13. 

NP 

A 

B 

C 

D 

14. 

207 

NP 

B 

C 

D 

A 

15. 

007 

P 

C 

D 

A 

B 

16. 

208 

NP 

D 

A 

B 

C 

17. 

209 

NP 

A 

B 

C 

D 

18. 

008 

P 

B 

C 

D 

A 

19. 

102 

P 

C 

D 

A 

B 

20. 

210 

NP 

D 

A 

B 

C 

Legend:  A 

B 
C 
D 

P 

NP 


Traditional  Design  without  color 
Prototype  Design  without  color 
Traditional  Design  with  color 
Prototype  Design  with  color 

Pilot 

Non-Pilot 
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APPENDIX  K:  EVALUATION  FORM 


^a^jdpanH^ 


Start  with  Traditional  Design  without  color:  A  □ 


Trails 

Attitude 

correct 

incorr. 

1  -  test  trial 

A 

usual 

pitch  15°  nose  up 

2  -  test  trial 

B 

usual 

bank  angle  25°  left 

3  -  test  trail 

C 

unusual 

bank  angle  60°  right 

4  -  test  trail 

D 

unusual 

pitch  15°  nose  down 

5  -  test  trail 

E 

unusual 

pitch  35°  nose  up  & 
bank  angle  55°  left 

6 

M 

unusual 

pitch  30°  nose  up 

7 

H 

usual 

bank  angle  35°  left 

8 

J 

usual 

pitch  10°  nose  up 
hank  angle  20°  right 

& 

9 

0 

unusual 

bank  angle  65°  left 

10 

G 

usual 

pitch  05°  nose  down 

11 

Q 

unusual 

pitch  05°  nose  down 
bank  angle  55°  left 

& 

12 

N 

unusual 

pitch  20°  nose  down 

13 

F 

usual 

pitch  20°  nose  up 

14 

P 

unusual 

bank  angle  55°  right 

15 

R 

unusual 

pitch  25°  nose  down 
bank  angle  35°  left 

& 

16 

K 

usual 

pitch  05°  nose  down 
bank  angle  35°  left 

& 

17 

I 

usual 

bank  angle  25°  right 

18 

S 

unusual 

pitch  40°  nose  up 
hank  angle  60°  right 

& 

19 

L 

usual 

pitch  5°  nose  down 
hank  angle  25°  right 

& 

Start  with  Traditional  Design  with  color:  C  □ 


Trails 

Attitude 

correct 

incorr. 

Time 

1  -  test  trial 

A 

usual 

pitch  15°  nose  up 

2  -  test  trial 

B 

usual 

bank  angle  25°  left 

3  -  test  trail 

C 

unusual 

bank  angle  60°  right 

4  -  test  trail 

D 

unusual 

pitch  15°  nose  down 

5  -  test  trail 

E 

unusual 

pitch  35°  nose  up  & 
bank  angle  55°  left 

6 

P 

unusual 

bank  angle  55°  right 

7 

M 

unusual 

pitch  30°  nose  up 

8 

K 

usual 

pitch  05°  nose  down 
bank  angle  35°  left 

& 

9 

N 

unusual 

pitch  20°  nose  down 

10 

I 

usual 

bank  angle  25°  right 

11 

J 

usual 

pitch  10°  nose  up 
hank  angle  20°  right 

& 

12 

0 

unusual 

bank  angle  65°  left 

13 

s 

unusual 

pitch  40°  nose  up 
bank  angle  60°  right 

& 

14 

G 

usual 

pitch  05°  nose  down 

15 

R 

unusual 

pitch  25°  nose  down 
bank  angle  35°  left 

& 

16 

H 

usual 

bank  angle  35°  left 

17 

Q 

unusual 

pitch  05°  nose  down 
bank  angle  55°  left 

& 

18 

L 

usual 

pitch  5°  nose  down 
bank  angle  25°  right 

& 

19 

F 

usual 

pitch  20°  nose  up 
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Start  with  WEBER-Box  without  color:  B  □ 


Trails 

Attitude 

correct 

incorr. 

Time 

1  -  test  trial 

A 

usual 

pitch  15°  nose  up 

2  -  test  trial 

B 

usual 

bank  angle  25°  left 

3  -  test  trail 

C 

unusual 

bank  angle  60°  right 

4  -  test  trail 

D 

unusual 

pitch  15°  nose  down 

5  -  test  trail 

E 

unusual 

pitch  35°  nose  up  & 
bank  angle  55°  left 

6 

I 

usual 

bank  angle  25°  right 

7 

G 

usual 

pitch  05°  nose  down 

8 

0 

unusual 

bank  angle  65°  left 

9 

P 

unusual 

bank  angle  55°  right 

10 

N 

unusual 

pitch  20°  nose  down 

11 

H 

usual 

bank  angle  35°  left 

12 

K 

usual 

pitch  05°  nose  down 
bank  angle  35°  left 

& 

13 

M 

unusual 

pitch  30°  nose  up 

14 

F 

usual 

pitch  20°  nose  up 

15 

Q 

unusual 

pitch  05°  nose  down 
bank  angle  55°  left 

& 

16 

J 

usual 

pitch  10°  nose  up 
bank  angle  20°  right 

& 

17 

R 

unusual 

pitch  25°  nose  down 
bank  angle  35°  left 

& 

18 

L 

usual 

pitch  5°  nose  down 
bank  angle  25°  right 

& 

19 

S 

unusual 

pitch  40°  nose  up 
hank  angle  60°  right 

& 
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Start  with  WEBER-Box  with  color:  D  □ 


Trails 

Attitude 

correct 

incorr. 

Time 

1  -  test  trial 

A 

usual 

pitch  15°  nose  up 

2  -  test  trial 

B 

usual 

bank  angle  25°  left 

3  -  test  trail 

C 

unusual 

bank  angle  60°  right 

4  -  test  trail 

D 

unusual 

pitch  15°  nose  down 

5  -  test  trail 

E 

unusual 

pitch  35°  nose  up  & 
bank  angle  55°  left 

6 

Q 

unusual 

pitch  05°  nose  down 
bank  angle  55°  left 

& 

7 

F 

usual 

pitch  20°  nose  up 

8 

K 

usual 

pitch  05°  nose  down 
bank  angle  35°  left 

& 

9 

0 

unusual 

bank  angle  65°  left 

10 

N 

unusual 

pitch  20°  nose  down 

11 

R 

unusual 

pitch  25°  nose  down 
bank  angle  35°  left 

& 

12 

L 

usual 

pitch  5°  nose  down 
bank  angle  25°  right 

& 

13 

I 

usual 

bank  angle  25°  right 

14 

M 

unusual 

pitch  30°  nose  up 

15 

J 

usual 

pitch  10°  nose  up 
bank  angle  20°  right 

& 

16 

G 

usual 

pitch  05°  nose  down 

17 

P 

unusual 

bank  angle  55°  right 

18 

S 

unusual 

pitch  40°  nose  up 
hank  angle  60°  right 

& 

19 

H 

usual 

bank  angle  35°  left 
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Legend: 


Index 

Attitude 

Pitch 

Bank  angle 

usual 

pitch  15°  nose  up 

B 

usual 

bank  angle  25°  left 

C 

unusual 

hank  angle  60°  right 

D 

unusual 

pitch  15°  nose  down 

E 

unusual 

pitch  35°  nose  up 

& 

bank  angle  55°  left 

F 

usual 

pitch  20°  nose  up 

G 

usual 

pitch  05°  nose  down 

H 

usual 

bank  angle  35°  left 

I 

usual 

bank  angle  25°  right 

J 

usual 

pitch  10°  nose  up 

& 

bank  angle  20°  right 

K 

usual 

pitch  5°  nose  down 

& 

bank  angle  35°  left 

L 

usual 

pitch  5°  nose  down 

& 

bank  angle  25°  right 

M 

unusual 

pitch  30°  nose  up 

N 

unusual 

pitch  20°  nose  down 

O 

unusual 

bank  angle  65°  left 

P 

unusual 

bank  angle  55°  right 

Q 

unusual 

pitch  5°  nose  down 

& 

hank  angle  55°  left 

R 

unusual 

pitch  25°  nose  down 

& 

bank  angle  35°  left 

S 

unusual 

pitch  40°  nose  up 

& 

bank  angle  60°  right 
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